Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate Resize() from pdata slice APIs #3573

Merged
merged 16 commits into from
Jul 13, 2021

Conversation

Aneurysm9
Copy link
Member

Description: Removes Resize() from the pdata slice APIs, replacing it with EnsureCapacity() and AppendEmptyN() methods. Many existing uses of Resize() can be directly replaced with AppendEmptyN() (which in turn calls EnsureCapacity()) to allocate new, empty slice members. Other uses are replaced with a single call to EnsureCapacity() followed by successive calls to AppendEmpty(), primarily when an upper bound on the number of slice elements is known but fewer may be used in the end. This avoids a second call to Resize() that had been necessary to remove the empty elements at the end of the list.

Link to tracking Issue: #2488

Testing: Existing tests were updated to reflect the new API.

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>
* Removed Resize()
* Added EnsureCapacity() and AppendEmptyN()

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>
@Aneurysm9 Aneurysm9 requested a review from alolita as a code owner July 6, 2021 22:30
@Aneurysm9 Aneurysm9 requested review from a team and bogdandrutu July 6, 2021 22:30
Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>
Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>
Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>
Copy link
Member

@tigrannajaryan tigrannajaryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the changes are fine, but replacement of Resize by RemoveIf is not an improvement. I am not sure this is net positive change. Perhaps only keep changes to AppendEmptyN (which are reasonable) but also keep Resize for the cases when we need to make the slice smaller?

processor/batchprocessor/splitlogs_test.go Show resolved Hide resolved
Comment on lines 123 to 125
dest.DoubleGauge().DataPoints().RemoveIf(func(_ pdata.DoubleDataPoint) bool {
return filterDestPoints()
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty awkward and harder to understand. Resize seemed to be not a bad idea?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, and I think this was the simpler one to understand. It would be sad to have to keep Resize around just for this, but I'm not sure there is a better option.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rewrote this to avoid copying the entire metric when its datapoints are to be split. I think this was the only place where using Resize() would still be attractive from a code reduction standpoint, but this implementation is probably still preferable.

@bogdandrutu
Copy link
Member

Most of the changes are fine, but replacement of Resize by RemoveIf is not an improvement. I am not sure this is net positive change. Perhaps only keep changes to AppendEmptyN (which are reasonable) but also keep Resize for the cases when we need to make the slice smaller?

I would say that we should apply the rule of minimal public API if possible, I do understand that these changes are not clear, but I would also say that right now we do something strange, we "copy" the entire metric then remove extra points, probably we should do the opposite and copy only the points we need to copy, so then no need to re-size.

@Aneurysm9 I have a question for you because I feel we can move even further and remove "AppendEmptyN" as well. What do you think about that as our initial API? I know it will add some extra overhead, but will allow us to think better about the API we really need and play with different possibilities?

My 2 cents when it comes to pdata API is if we can keep it as small as possible we will be able to do more optimizations later, otherwise we may be blocked by the public APIs we expose right now.

@Aneurysm9
Copy link
Member Author

Most of the changes are fine, but replacement of Resize by RemoveIf is not an improvement. I am not sure this is net positive change. Perhaps only keep changes to AppendEmptyN (which are reasonable) but also keep Resize for the cases when we need to make the slice smaller?

I would say that we should apply the rule of minimal public API if possible, I do understand that these changes are not clear, but I would also say that right now we do something strange, we "copy" the entire metric then remove extra points, probably we should do the opposite and copy only the points we need to copy, so then no need to re-size.

I'll take another pass at this. I had saved the split functions for last since they were clearly different and I couldn't really think of a better way to handle them at the time. Maybe tomorrow I'll see it with fresh eyes.

@Aneurysm9 I have a question for you because I feel we can move even further and remove "AppendEmptyN" as well. What do you think about that as our initial API? I know it will add some extra overhead, but will allow us to think better about the API we really need and play with different possibilities?

My 2 cents when it comes to pdata API is if we can keep it as small as possible we will be able to do more optimizations later, otherwise we may be blocked by the public APIs we expose right now.

Definitely agree that minimizing the API is a good goal and AppendEmptyN can be eliminated since it is basically EnsureCapacity+AppendEmpty. I included it because it simplified the conversion for a common usage pattern of Resize.

@bogdandrutu
Copy link
Member

@Aneurysm9 any update on this?

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>
@Aneurysm9
Copy link
Member Author

@Aneurysm9 any update on this?

This is my current task. AppendEmptyN() eliminated with extreme prejudice, now on to that pesky splitting logic.

…unless necessary

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>
Copy link
Member

@bogdandrutu bogdandrutu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Aneurysm9 in order to be able to merge this (since we are close to the release date, and avoid waiting until the release), I would suggest to keep "Resize" as deprecated in this PR - only the function we can remove the tests. This way contrib will not break and we can do the contrib upgrade before or after the release.

What do you think?

@Aneurysm9
Copy link
Member Author

@Aneurysm9 in order to be able to merge this (since we are close to the release date, and avoid waiting until the release), I would suggest to keep "Resize" as deprecated in this PR - only the function we can remove the tests. This way contrib will not break and we can do the contrib upgrade before or after the release.

What do you think?

That seems reasonable. I was having trouble getting contrib to build with rewrites for core, so I'm not sure how much will actually be impacted and taking a two-step approach to removing Resize is probably safest.

@bogdandrutu
Copy link
Member

Please rebase :) sorry

@bogdandrutu
Copy link
Member

@Aneurysm9 I gave a try to resolve conflicts from the UI, let's see what i've done :))

@Aneurysm9
Copy link
Member Author

@bogdandrutu I think the contrib tests are having a struggle because /model is now a separate module but the otel-from-tree target doesn't set up replace directives for it. The explosion of replace directives to manage is one of the reasons why we don't try to keep the Go contrib repo constantly up-to-date and instead just update it when releases are made.

@bogdandrutu bogdandrutu changed the title Remove Resize() from pdata slice APIs Deprecate Resize() from pdata slice APIs Jul 13, 2021
@bogdandrutu bogdandrutu merged commit 9e8bded into open-telemetry:main Jul 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants