Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API from ParquetExec to ParquetExecBuilder #12799

Merged
merged 5 commits into from
Oct 8, 2024

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Oct 7, 2024

Which issue does this PR close?

Closes #12737
Closes #12726

Rationale for this change

See #12737 for rationale (basically many times we manipulate a ParquetExec downstream we end up needing to create a builder from an existing ParquetExec

What changes are included in this PR?

Changes:

  1. Add ParquetExec::into_builder() method
  2. Add docs with examples of usage
  3. Update some existing code to use it

Are these changes tested?

Yes

Are there any user-facing changes?

New API and Docs

@alamb
Copy link
Contributor Author

alamb commented Oct 7, 2024

fyi @NGA-TRAN

@alamb alamb changed the title API to go from ParquetExec to ParquetExecBuilder API from ParquetExec to ParquetExecBuilder Oct 7, 2024
@github-actions github-actions bot added the core Core DataFusion crate label Oct 7, 2024
@@ -479,9 +544,15 @@ impl ParquetExec {
self.pruning_predicate.as_ref()
}

/// return the optional file reader factory
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this API for completeness

Copy link
Contributor

@NGA-TRAN NGA-TRAN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Thanks @alamb

Copy link
Contributor

@jayzhan211 jayzhan211 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@alamb
Copy link
Contributor Author

alamb commented Oct 8, 2024

Thanks again @jayzhan211 and @NGA-TRAN

@alamb alamb merged commit 9df957b into apache:main Oct 8, 2024
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Easier way to convert between ParquetExec and ParquetExecBuilder
3 participants