Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of Spark-compatible decimal aggregates #951

Open
Tracked by #391
andygrove opened this issue Sep 18, 2024 · 1 comment
Open
Tracked by #391

Improve performance of Spark-compatible decimal aggregates #951

andygrove opened this issue Sep 18, 2024 · 1 comment
Labels
enhancement New feature or request performance
Milestone

Comments

@andygrove
Copy link
Member

andygrove commented Sep 18, 2024

What is the problem the feature request solves?

The benchmarks added in #948 show that Comet's Spark-compatible aggregates are ~50% slower than the DataFusion equivalents:

aggregate/avg_decimal_datafusion
                        time:   [653.56 µs 657.57 µs 662.06 µs]
aggregate/avg_decimal_comet
                        time:   [1.0581 ms 1.0592 ms 1.0604 ms]
aggregate/sum_decimal_datafusion
                        time:   [695.51 µs 696.48 µs 697.60 µs]
aggregate/sum_decimal_comet
                        time:   [1.0218 ms 1.0230 ms 1.0242 ms]

Describe the potential solution

No response

Additional context

No response

@andygrove
Copy link
Member Author

Related upstream changes in arrow-rs: apache/arrow-rs#6419

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance
Projects
None yet
Development

No branches or pull requests

1 participant