Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Arrow columnar format v1.4 #5326

Closed
1 of 3 tasks
spencerwilson opened this issue Jan 23, 2024 · 4 comments
Closed
1 of 3 tasks

Support Arrow columnar format v1.4 #5326

spencerwilson opened this issue Jan 23, 2024 · 4 comments
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog

Comments

@spencerwilson
Copy link

spencerwilson commented Jan 23, 2024

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

My initial problem was "I'm not sure what version of the Arrow spec(s) the arrow crate complies with". Documenting the level of support would be great.

After some searching it seems that the arrow crate presently supports "Arrow columnar format version 1.3", as defined here: https://github.com/apache/arrow/blob/3fe598ae4dfd7805ab05452dd5ed4b0d6c97d8d5/format/Schema.fbs#L20-L26

I'm creating this tracking issue since I didn't see anything about 1.4 by trying obvious searches in GitHub issue search. To be clear, I'm not suffering from a lack of support for 1.4 features currently.

Describe the solution you'd like

Support everything in Arrow columnar format version 1.4. Based on

This would include support for

Describe alternatives you've considered

Declare an intention not to support things from 1.4, giving some rationale.

Additional context
n/a

@spencerwilson spencerwilson added the enhancement Any new improvement worthy of a entry in the changelog label Jan 23, 2024
@tustvold
Copy link
Contributor

FWIW we don't currently support the array view types for which this functionality was added - #4253

Always happy to review documentation PRs

@alamb
Copy link
Contributor

alamb commented Feb 8, 2024

Filed #5374 to track adding StringViewArray and #5375 to track adding ListViews and I updated the description on this ticket

@spencerwilson
Copy link
Author

Similarly, I added a reference to #3520 , where the RunEndEncoded type was actually designated as part of columnar format v1.3.

@tustvold
Copy link
Contributor

Closed by #5502, follow on work to flesh out support for ListView and StringViewArray are tracked in #5375 and #5374

@tustvold tustvold added the arrow Changes to the arrow crate label Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Development

No branches or pull requests

3 participants