You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We rename InputStream.h/cpp to SeeableInputStream.h/cpp to differentiate from velox/dwio/common/InputStream.h/cpp
PagedInputStream is used for compressed ORC and DWRF files, and it needs to handle the ORC compression format. We will leave it in the dwrf namespace for now.
But si was not used other than one test velox/dwio/dwrf/test/CacheInputTest.cpp in makeStripeData(). Since StreamIdentifier and EncodingKey not related to Parquet but just DWRF and ORC, we will change this parameter to be some common type.
The text was updated successfully, but these errors were encountered:
yingsu00
changed the title
[Refactoring] Move SeekableInputStream, BufferredInput, (de)compressor related classes to velox::dwio::common::io
[Refactoring] Move SeekableInputStream, BufferredInput, (de)compressor related classes to dwio::common::io
May 14, 2022
…#1618)
What changes were proposed in this pull request?
add format check for ch backend
(Fixes: facebookincubator#1612)
How was this patch tested?
unit tests
From #1533 we mentioned the SeekableInputStream related classes shall be moved. This issue is to track the move itself.
1.
SeekableInputStream
and its subclasses (exceptPagedInputStream
at this time) will be moved.We rename
InputStream.h/cpp
toSeeableInputStream.h/cpp
to differentiate fromvelox/dwio/common/InputStream.h/cpp
PagedInputStream
is used for compressed ORC and DWRF files, and it needs to handle the ORC compression format. We will leave it in the dwrf namespace for now.2. Compressor and subclasses
velox/dwio/dwrf/common/Compression.h/cpp. == > velox/dwio/common/Compression.h/cpp
3. BufferedInput and CachedBufferedInput
namespace facebook::velox::dwrf. ==> facebook::velox::dwio::common::io
file velox/dwio/dwrf/common/BufferedInput.h/cpp. == > velox/dwio/common/io/BufferedInput.h/cpp.
file velox/dwio/dwrf/common/CachedBufferedInput.h/cpp. == > velox/dwio/common/io/CachedBufferedInput.h/cpp.
BufferedInput.h needs to reference
StreamIdentifier
in Common.h.But
si
was not used other than one test velox/dwio/dwrf/test/CacheInputTest.cpp in makeStripeData(). Since StreamIdentifier and EncodingKey not related to Parquet but just DWRF and ORC, we will change this parameter to be some common type.The text was updated successfully, but these errors were encountered: