-
Notifications
You must be signed in to change notification settings - Fork 5.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Restructure and expand presto_cpp docs
- Loading branch information
1 parent
e344635
commit fccc180
Showing
7 changed files
with
107 additions
and
84 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
19 changes: 0 additions & 19 deletions
19
presto-docs/src/main/sphinx/prestissimo/prestissimo-limitations.rst
This file was deleted.
Oops, something went wrong.
28 changes: 0 additions & 28 deletions
28
presto-docs/src/main/sphinx/prestissimo/prestissimo-properties.rst
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
********** | ||
Presto C++ | ||
********** | ||
|
||
Note: Presto C++ is in active development. See :doc:`Limitations </presto_cpp/limitations>`. | ||
|
||
.. toctree:: | ||
:maxdepth: 1 | ||
|
||
presto_cpp/features | ||
presto_cpp/limitations | ||
|
||
Overview | ||
======== | ||
|
||
Presto C++, sometimes referred to by the development name Prestissimo, is a | ||
drop-in replacement for Presto workers written in C++ and based on the | ||
`Velox <https://velox-lib.io/>`_ library. | ||
It implements the same RESTful endpoints as Java workers using the Proxygen C++ | ||
HTTP framework. | ||
Because communication with the Java coordinator and across workers is only | ||
done using the REST endpoints, Presto C++ does not use JNI and does not | ||
require a JVM on worker nodes. | ||
|
||
Presto C++'s codebase is located at `presto-native-execution | ||
<https://github.com/prestodb/presto/tree/master/presto-native-execution>`_. | ||
|
||
Motivation and Vision | ||
===================== | ||
|
||
Presto aims to be the top performing system for data lakes. | ||
To achieve this goal, the Presto community is moving the Presto | ||
evaluation engine from the native Java-based implementation to a new | ||
implementation written in C++ using `Velox <https://velox-lib.io/>`_. | ||
|
||
By moving the evaluation engine to a library, the intent is to enable the | ||
Presto community to focus on more features and better integration with table | ||
formats and other data warehousing systems. | ||
|
||
Supported Use Cases | ||
=================== | ||
|
||
Only specific connectors are supported in the Presto C++ evaluation engine. | ||
|
||
* Hive connector for reads and writes, including CTAS, are supported. | ||
|
||
* Iceberg tables are supported only for reads. | ||
|
||
* Iceberg connector supports both V1 and V2 tables, including tables with delete files. | ||
|
||
* TPCH connector, with ``tpch.naming=standard`` catalog property. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
====================== | ||
Presto C++ Limitations | ||
====================== | ||
|
||
.. contents:: | ||
:local: | ||
:backlinks: none | ||
:depth: 1 | ||
|
||
General Limitations | ||
=================== | ||
|
||
The C++ evaluation engine has a number of limitations: | ||
|
||
* Not all built-in functions are implemented in C++. Attempting to use unimplemented functions results in a query failure. For supported functions, see `Function Coverage <https://facebookincubator.github.io/velox/functions/presto/coverage.html>`_. | ||
|
||
* Not all built-in types are implemented in C++. Attempting to use unimplemented types will result in a query failure. | ||
|
||
* Certain parts of the plugin SPI are not used by the C++ evaluation engine. In particular, C++ workers will not load any plugin in the plugins directory, and certain plugin types are either partially or completely unsupported. | ||
|
||
* ``PageSourceProvider``, ``RecordSetProvider``, and ``PageSinkProvider`` do not work in the C++ evaluation engine. | ||
|
||
* User-supplied functions, types, parametric types and block encodings are not supported. | ||
|
||
* The event listener plugin does not work at the split level. | ||
|
||
* User-defined functions do not work in the same way, see `Remote Function Execution <features.html#remote-function-execution>`_. | ||
|
||
* Memory management works differently in the C++ evaluation engine. In particular: | ||
|
||
* The OOM killer is not supported. | ||
* The reserved pool is not supported. | ||
* In general, queries may use more memory than they are allowed to through memory arbitration. See `Memory Management <https://facebookincubator.github.io/velox/develop/memory.html>`_. | ||
|
||
Functions | ||
========= | ||
|
||
reduce_agg | ||
---------- | ||
|
||
In C++ based Presto, ``reduce_agg`` is not permitted to return ``null`` in either the | ||
``inputFunction`` or the ``combineFunction``. In Presto (Java), this is permitted | ||
but undefined behavior. For more information about ``reduce_agg`` in Presto, | ||
see `reduce_agg <../functions/aggregate.html#reduce_agg>`_. |