Add support for DECIMAL types to Simple Function API #9096

mbasmanova · 2024-03-15T16:03:57Z

Use the new functionality to re-write decimal plus, minus, multiple, divide, between, negate, floor and round.

type/Type.h

Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration.

Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration.

  registerFunction<
      DecimalAddFunction,
      LongDecimal<P3, S3>,
      LongDecimal<P1, S1>,
      LongDecimal<P2, S2>>({"plus"}, constraints);

expression/UdfTypeResolver.h

Define arg_type and out_type for LongDecimal and ShortDecimal.

	arg_type<LongDecimal> = int128_t
	out_type<LongDecimal> = int128_t

	arg_type<ShortDecimal> = int64_t
	out_type<ShortDecimal> = int64_t

functions/Registerer.h

Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types.

template <template <class> typename Func, typename TReturn, typename... TArgs>
void registerFunction(
    const std::vector<std::string>& aliases = {},
    const std::vector<exec::SignatureVariable>& constraints = {})

Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal).

  std::vector<exec::SignatureVariable> constraints = {
      exec::SignatureVariable(
          P3::name(),
          fmt::format(
              "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)",
              fmt::arg("a_precision", P1::name()),
              fmt::arg("b_precision", P2::name()),
              fmt::arg("a_scale", S1::name()),
              fmt::arg("b_scale", S2::name())),
          exec::ParameterType::kIntegerParameter),
      exec::SignatureVariable(
          S3::name(),
          fmt::format(
              "max({a_scale}, {b_scale})",
              fmt::arg("a_scale", S1::name()),
              fmt::arg("b_scale", S2::name())),
          exec::ParameterType::kIntegerParameter),
  };

core/SimpleFunctionMetadata.h

Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature.

Decimal “plus” function has a single signature:

(decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3)

But 5 implementations:

	(int64_t, int64_t) -> int64_t
	(int64_t, int64_t) -> int128_t
	(int64_t, int128_t) -> int128_t
	(int128_t, int64_t) -> int128_t
	(int128_t, int128_t) -> int128_t

We need a way to distinguish between these.

expression/SimpleFunctionRegistry.h/cpp

Allow for storing multiple implementations for a single signature.

using SignatureMap = std::unordered_map<
    FunctionSignature,
    std::vector<std::unique_ptr<const FunctionEntry>>>;
using FunctionMap = std::unordered_map<std::string, SignatureMap>;

Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type.

core/SimpleFunctionMetadata.h

Introduce optional initializeTypes method for a function to receive input types. Functions that operate on decimal types use this method to get access to precision and scale of the arguments.

void initializeTypes(const std::vector<TypePtr>& argTypes)

Example: Decimal ADD

Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale.

template <typename TExec>
struct DecimalAddFunction {
  VELOX_DEFINE_FUNCTION_TYPES(TExec);

  void initializeTypes(const std::vector<TypePtr>& argTypes) {
    auto aType = argTypes[0];
    auto bType = argTypes[1];
    auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType);
    auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType);
    auto [rPrecision, rScale] = Addition::computeResultPrecisionScale(
        aPrecision, aScale, bPrecision, bScale);
    aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale);
    bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale);
  }

  template <typename R, typename A, typename B>
  void call(R& out, const A& a, const B& b) {
    Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_);
  }

 private:
  uint8_t aRescale_;
  uint8_t bRescale_;
};

The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result.

  std::vector<exec::SignatureVariable> constraints = {
      exec::SignatureVariable(
          P3::name(),
          fmt::format(
              "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)",
              fmt::arg("a_precision", P1::name()),
              fmt::arg("b_precision", P2::name()),
              fmt::arg("a_scale", S1::name()),
              fmt::arg("b_scale", S2::name())),
          exec::ParameterType::kIntegerParameter),
      exec::SignatureVariable(
          S3::name(),
          fmt::format(
              "max({a_scale}, {b_scale})",
              fmt::arg("a_scale", S1::name()),
              fmt::arg("b_scale", S2::name())),
          exec::ParameterType::kIntegerParameter),
  };

  // (long, long) -> long
  registerFunction<
      DecimalAddFunction,
      LongDecimal<P3, S3>,
      LongDecimal<P1, S1>,
      LongDecimal<P2, S2>>({"plus"}, constraints);

  // (short, short) -> short
  registerFunction<
      DecimalAddFunction,
      ShortDecimal<P3, S3>,
      ShortDecimal<P1, S1>,
      ShortDecimal<P2, S2>>({"plus"}, constraints);

  // (short, short) -> long
  registerFunction<
      DecimalAddFunction,
      LongDecimal<P3, S3>,
      ShortDecimal<P1, S1>,
      ShortDecimal<P2, S2>>({"plus"}, constraints);

  // (short, long) -> long
  registerFunction<
      DecimalAddFunction,
      LongDecimal<P3, S3>,
      ShortDecimal<P1, S1>,
      LongDecimal<P2, S2>>({"plus"}, constraints);

  // (long, short) -> long
  registerFunction<
      DecimalAddFunction,
      LongDecimal<P3, S3>,
      LongDecimal<P1, S1>,
      ShortDecimal<P2, S2>>({"plus"}, constraints);

Differential Revision: D54953663

facebook-github-bot · 2024-03-15T16:04:14Z

This pull request was exported from Phabricator. Differential Revision: D54953663

netlify · 2024-03-15T16:04:15Z

✅ Deploy Preview for meta-velox canceled.

Name	Link
🔨 Latest commit	`4ea35af`
🔍 Latest deploy log	https://app.netlify.com/sites/meta-velox/deploys/65f9e06486d5030008a34feb

…or#9096) Summary: **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Introduce optional initializeTypes method for a function to receive input types. Functions that operate on decimal types use this method to get access to precision and scale of the arguments. ``` void initializeTypes(const std::vector<TypePtr>& argTypes) ``` Differential Revision: D54953663

facebook-github-bot · 2024-03-15T16:27:16Z

This pull request was exported from Phabricator. Differential Revision: D54953663

…or#9096) Summary: **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Introduce optional initializeTypes method for a function to receive input types. Functions that operate on decimal types use this method to get access to precision and scale of the arguments. ``` void initializeTypes(const std::vector<TypePtr>& argTypes) ``` **Example: Decimal ADD** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalAddFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); void initializeTypes(const std::vector<TypePtr>& argTypes) { auto aType = argTypes[0]; auto bType = argTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Differential Revision: D54953663

facebook-github-bot · 2024-03-15T17:31:18Z

This pull request was exported from Phabricator. Differential Revision: D54953663

…or#9096) Summary: **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Introduce optional initializeTypes method for a function to receive input types. Functions that operate on decimal types use this method to get access to precision and scale of the arguments. ``` void initializeTypes(const std::vector<TypePtr>& argTypes) ``` **Example: Decimal ADD** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalAddFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); void initializeTypes(const std::vector<TypePtr>& argTypes) { auto aType = argTypes[0]; auto bType = argTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Differential Revision: D54953663

facebook-github-bot · 2024-03-15T17:32:18Z

This pull request was exported from Phabricator. Differential Revision: D54953663

…or#9096) Summary: **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Introduce optional initializeTypes method for a function to receive input types. Functions that operate on decimal types use this method to get access to precision and scale of the arguments. ``` void initializeTypes(const std::vector<TypePtr>& argTypes) ``` **Example: Decimal ADD** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalAddFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); void initializeTypes(const std::vector<TypePtr>& argTypes) { auto aType = argTypes[0]; auto bType = argTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Differential Revision: D54953663

facebook-github-bot · 2024-03-15T17:49:48Z

This pull request was exported from Phabricator. Differential Revision: D54953663

…or#9096) Summary: **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Introduce optional initializeTypes method for a function to receive input types. Functions that operate on decimal types use this method to get access to precision and scale of the arguments. ``` void initializeTypes(const std::vector<TypePtr>& argTypes) ``` **Example: Decimal ADD** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalAddFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); void initializeTypes(const std::vector<TypePtr>& argTypes) { auto aType = argTypes[0]; auto bType = argTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Differential Revision: D54953663

facebook-github-bot · 2024-03-15T18:11:18Z

This pull request was exported from Phabricator. Differential Revision: D54953663

…or#9096) Summary: **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Introduce optional initializeTypes method for a function to receive input types. Functions that operate on decimal types use this method to get access to precision and scale of the arguments. ``` void initializeTypes(const std::vector<TypePtr>& argTypes) ``` **Example: Decimal ADD** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalAddFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); void initializeTypes(const std::vector<TypePtr>& argTypes) { auto aType = argTypes[0]; auto bType = argTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Differential Revision: D54953663

facebook-github-bot · 2024-03-15T18:12:24Z

This pull request was exported from Phabricator. Differential Revision: D54953663

…or#9096) Summary: **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Introduce optional initializeTypes method for a function to receive input types. Functions that operate on decimal types use this method to get access to precision and scale of the arguments. ``` void initializeTypes(const std::vector<TypePtr>& argTypes) ``` **Example: Decimal ADD** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalAddFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); void initializeTypes(const std::vector<TypePtr>& argTypes) { auto aType = argTypes[0]; auto bType = argTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Differential Revision: D54953663

facebook-github-bot · 2024-03-18T06:14:13Z

This pull request was exported from Phabricator. Differential Revision: D54953663

…or#9096) Summary: Use the new functionality to re-write decimal plus, minus, multiple, divide, between, negate, floor and round. **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Add 'inputTypes' parameter to 'initialize' method. Functions that operate on decimal types use this parameter to get access to precision and scale of the arguments. Landed separately. ``` void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& config, ...) ``` **Example: Decimal Plus** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalPlusFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); template <typename A, typename B> void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& /*config*/, A* /*a*/, B* /*b*/) { auto aType = inputTypes[0]; auto bType = inputTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Pull Request resolved: facebookincubator#9096 Reviewed By: xiaoxmeng Differential Revision: D54953663

facebook-github-bot · 2024-03-19T18:10:28Z

This pull request was exported from Phabricator. Differential Revision: D54953663

…or#9096) Summary: Use the new functionality to re-write decimal plus, minus, multiple, divide, between, negate, floor and round. **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Add 'inputTypes' parameter to 'initialize' method. Functions that operate on decimal types use this parameter to get access to precision and scale of the arguments. Landed separately. ``` void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& config, ...) ``` **Example: Decimal Plus** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalPlusFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); template <typename A, typename B> void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& /*config*/, A* /*a*/, B* /*b*/) { auto aType = inputTypes[0]; auto bType = inputTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Pull Request resolved: facebookincubator#9096 Reviewed By: xiaoxmeng Differential Revision: D54953663

facebook-github-bot · 2024-03-19T18:51:25Z

This pull request was exported from Phabricator. Differential Revision: D54953663

…or#9096) Summary: Use the new functionality to re-write decimal plus, minus, multiple, divide, between, negate, floor and round. **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Add 'inputTypes' parameter to 'initialize' method. Functions that operate on decimal types use this parameter to get access to precision and scale of the arguments. Landed separately. ``` void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& config, ...) ``` **Example: Decimal Plus** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalPlusFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); template <typename A, typename B> void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& /*config*/, A* /*a*/, B* /*b*/) { auto aType = inputTypes[0]; auto bType = inputTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Pull Request resolved: facebookincubator#9096 Reviewed By: xiaoxmeng Differential Revision: D54953663

facebook-github-bot · 2024-03-19T18:58:42Z

This pull request was exported from Phabricator. Differential Revision: D54953663

facebook-github-bot · 2024-03-19T23:17:17Z

This pull request has been merged in 9c7eb6c.

mbasmanova · 2024-03-22T17:40:13Z

CC: @laithsakka

facebookincubator#9120) Summary: Pull Request resolved: facebookincubator#9120 This change is part of enabling simple functions to process inputs of decimal type. Such processing requires access to decimal type parameters (precision and scale). This change provides full type information to the function constructor. A follow-up change will pass this to simple function's 'initialize' method. See facebookincubator#9096 for the end-to-end workflow. Reviewed By: xiaoxmeng Differential Revision: D55011267 fbshipit-source-id: 323e39aec04e0fd6d0db4b4be4bfdf4bd9544dd0

…cubator#9124) Summary: Pull Request resolved: facebookincubator#9124 This change is part of enabling simple functions to process inputs of decimal type. Such processing requires access to decimal type parameters (precision and scale). This change provides full type information via 'initialize' method. See facebookincubator#9096 for the end-to-end workflow. Reviewed By: xiaoxmeng Differential Revision: D55012189 fbshipit-source-id: 95d5d05054730a41788a4fac51fe78f77c7bfac2

…or#9096) Summary: Use the new functionality to re-write decimal plus, minus, multiple, divide, between, negate, floor and round. **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Add 'inputTypes' parameter to 'initialize' method. Functions that operate on decimal types use this parameter to get access to precision and scale of the arguments. Landed separately. ``` void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& config, ...) ``` **Example: Decimal Plus** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalPlusFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); template <typename A, typename B> void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& /*config*/, A* /*a*/, B* /*b*/) { auto aType = inputTypes[0]; auto bType = inputTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Pull Request resolved: facebookincubator#9096 Reviewed By: xiaoxmeng Differential Revision: D54953663 fbshipit-source-id: 96377a01090249b21d1f4ed6f825cf8b1e88df2f

…cubator#9124) Summary: Pull Request resolved: facebookincubator#9124 This change is part of enabling simple functions to process inputs of decimal type. Such processing requires access to decimal type parameters (precision and scale). This change provides full type information via 'initialize' method. See facebookincubator#9096 for the end-to-end workflow. Reviewed By: xiaoxmeng Differential Revision: D55012189 fbshipit-source-id: 95d5d05054730a41788a4fac51fe78f77c7bfac2

…or#9096) Summary: Use the new functionality to re-write decimal plus, minus, multiple, divide, between, negate, floor and round. **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Add 'inputTypes' parameter to 'initialize' method. Functions that operate on decimal types use this parameter to get access to precision and scale of the arguments. Landed separately. ``` void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& config, ...) ``` **Example: Decimal Plus** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalPlusFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); template <typename A, typename B> void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& /*config*/, A* /*a*/, B* /*b*/) { auto aType = inputTypes[0]; auto bType = inputTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Pull Request resolved: facebookincubator#9096 Reviewed By: xiaoxmeng Differential Revision: D54953663 fbshipit-source-id: 96377a01090249b21d1f4ed6f825cf8b1e88df2f

…cubator#9124) Summary: Pull Request resolved: facebookincubator#9124 This change is part of enabling simple functions to process inputs of decimal type. Such processing requires access to decimal type parameters (precision and scale). This change provides full type information via 'initialize' method. See facebookincubator#9096 for the end-to-end workflow. Reviewed By: xiaoxmeng Differential Revision: D55012189 fbshipit-source-id: 95d5d05054730a41788a4fac51fe78f77c7bfac2

…or#9096) Summary: Use the new functionality to re-write decimal plus, minus, multiple, divide, between, negate, floor and round. **type/Type.h** Add P1, P2, P3, P4, S1, S2, S3, S4 types to specify precision and scale parameters for decimal types during function registration. Add LongDecimal<P, S> and ShortDecimal<P, S> templates to specify decimal argument and return types during function registration. ``` registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); ``` **expression/UdfTypeResolver.h** Define arg_type and out_type for LongDecimal and ShortDecimal. ``` arg_type<LongDecimal> = int128_t out_type<LongDecimal> = int128_t arg_type<ShortDecimal> = int64_t out_type<ShortDecimal> = int64_t ``` **functions/Registerer.h** Add optional ‘constraints’ parameter to registerFunction template. This allows to specify rules for calculating precision and scale for decimal return types. ``` template <template <class> typename Func, typename TReturn, typename... TArgs> void registerFunction( const std::vector<std::string>& aliases = {}, const std::vector<exec::SignatureVariable>& constraints = {}) ``` Here is how we can specify calculation of precision and scale for the return type of plus(decimal, decimal). ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; ``` **core/SimpleFunctionMetadata.h** Extend SimpleFunctionMetadata to store physical types (TypeKind) of input arguments and return type in addition to signature. Decimal “plus” function has a single signature: ``` (decimal(p1, s1), decimal(p2, s2)) -> decimal(p3, s3) ``` But 5 implementations: ``` (int64_t, int64_t) -> int64_t (int64_t, int64_t) -> int128_t (int64_t, int128_t) -> int128_t (int128_t, int64_t) -> int128_t (int128_t, int128_t) -> int128_t ``` We need a way to distinguish between these. **expression/SimpleFunctionRegistry.h/cpp** Allow for storing multiple implementations for a single signature. ``` using SignatureMap = std::unordered_map< FunctionSignature, std::vector<std::unique_ptr<const FunctionEntry>>>; using FunctionMap = std::unordered_map<std::string, SignatureMap>; ``` Modify SimpleFunctionRegistry::resolveFunction method to find an implementation with matching signature and matching TypeKinds for arguments and return type. **core/SimpleFunctionMetadata.h** Add 'inputTypes' parameter to 'initialize' method. Functions that operate on decimal types use this parameter to get access to precision and scale of the arguments. Landed separately. ``` void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& config, ...) ``` **Example: Decimal Plus** Here is how a function that adds 2 decimal numbers can be defined. This function supports adding decimal numbers with possibly different precision and scale. ``` template <typename TExec> struct DecimalPlusFunction { VELOX_DEFINE_FUNCTION_TYPES(TExec); template <typename A, typename B> void initialize( const std::vector<TypePtr>& inputTypes, const core::QueryConfig& /*config*/, A* /*a*/, B* /*b*/) { auto aType = inputTypes[0]; auto bType = inputTypes[1]; auto [aPrecision, aScale] = getDecimalPrecisionScale(*aType); auto [bPrecision, bScale] = getDecimalPrecisionScale(*bType); auto [rPrecision, rScale] = Addition::computeResultPrecisionScale( aPrecision, aScale, bPrecision, bScale); aRescale_ = Addition::computeRescaleFactor(aScale, bScale, rScale); bRescale_ = Addition::computeRescaleFactor(bScale, aScale, rScale); } template <typename R, typename A, typename B> void call(R& out, const A& a, const B& b) { Addition::template apply<R, A, B>(out, a, b, aRescale_, bRescale_); } private: uint8_t aRescale_; uint8_t bRescale_; }; ``` The registration involves specifying a rule for calculating precision and scale for the result based on precision and scale of the inputs and provides 5 implementations with all possible permutations of short and long decimals in the input and result. ``` std::vector<exec::SignatureVariable> constraints = { exec::SignatureVariable( P3::name(), fmt::format( "min(38, max({a_precision} - {a_scale}, {b_precision} - {b_scale}) + max({a_scale}, {b_scale}) + 1)", fmt::arg("a_precision", P1::name()), fmt::arg("b_precision", P2::name()), fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), exec::SignatureVariable( S3::name(), fmt::format( "max({a_scale}, {b_scale})", fmt::arg("a_scale", S1::name()), fmt::arg("b_scale", S2::name())), exec::ParameterType::kIntegerParameter), }; // (long, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> short registerFunction< DecimalAddFunction, ShortDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); // (short, long) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, ShortDecimal<P1, S1>, LongDecimal<P2, S2>>({"plus"}, constraints); // (long, short) -> long registerFunction< DecimalAddFunction, LongDecimal<P3, S3>, LongDecimal<P1, S1>, ShortDecimal<P2, S2>>({"plus"}, constraints); ``` Pull Request resolved: facebookincubator#9096 Reviewed By: xiaoxmeng Differential Revision: D54953663 fbshipit-source-id: 96377a01090249b21d1f4ed6f825cf8b1e88df2f

…cubator#9124) Summary: Pull Request resolved: facebookincubator#9124 This change is part of enabling simple functions to process inputs of decimal type. Such processing requires access to decimal type parameters (precision and scale). This change provides full type information via 'initialize' method. See facebookincubator#9096 for the end-to-end workflow. Reviewed By: xiaoxmeng Differential Revision: D55012189 fbshipit-source-id: 95d5d05054730a41788a4fac51fe78f77c7bfac2

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 15, 2024

facebook-github-bot added the fb-exported label Mar 15, 2024

mbasmanova requested a review from majetideepak March 15, 2024 16:05

mbasmanova mentioned this pull request Mar 15, 2024

Add support for DECIMAL types to Simple Function API #9097

Closed

mbasmanova requested review from Yuhta, kagamiori and xiaoxmeng March 15, 2024 16:07

mbasmanova force-pushed the export-D54953663 branch from ee58a89 to a494c6e Compare March 15, 2024 16:27

mbasmanova force-pushed the export-D54953663 branch from a494c6e to 3853efb Compare March 15, 2024 17:31

mbasmanova force-pushed the export-D54953663 branch from 3853efb to d48298a Compare March 15, 2024 17:32

mbasmanova force-pushed the export-D54953663 branch from d48298a to 4130ffd Compare March 15, 2024 17:49

mbasmanova force-pushed the export-D54953663 branch from 4130ffd to 3e1a840 Compare March 15, 2024 18:11

mbasmanova force-pushed the export-D54953663 branch from 3e1a840 to b93f785 Compare March 15, 2024 18:12

mbasmanova force-pushed the export-D54953663 branch from b93f785 to 82dc279 Compare March 18, 2024 06:14

mbasmanova force-pushed the export-D54953663 branch from 9003b89 to 7e68798 Compare March 19, 2024 17:33

mbasmanova force-pushed the export-D54953663 branch from 7e68798 to faf80f3 Compare March 19, 2024 18:10

mbasmanova force-pushed the export-D54953663 branch from faf80f3 to 630a1d6 Compare March 19, 2024 18:51

mbasmanova force-pushed the export-D54953663 branch from 630a1d6 to 4ea35af Compare March 19, 2024 18:58

facebook-github-bot closed this in 9c7eb6c Mar 19, 2024

facebook-github-bot added the Merged label Mar 19, 2024

mbasmanova mentioned this pull request Mar 29, 2024

Rewrite Presto function greatest/least using simple function interface #3728

Closed

Real-Chen-Happy mentioned this pull request Mar 30, 2024

Refactor greatest and least Presto functions using simple function API #9308

Closed

mbasmanova mentioned this pull request Jul 18, 2024

Change FunctionSignature returnType to std::optional to support the return type varied with config #10467

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for DECIMAL types to Simple Function API #9096

Add support for DECIMAL types to Simple Function API #9096

mbasmanova commented Mar 15, 2024 •

edited

Loading

facebook-github-bot commented Mar 15, 2024

netlify bot commented Mar 15, 2024 •

edited

Loading

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 18, 2024

facebook-github-bot commented Mar 19, 2024

facebook-github-bot commented Mar 19, 2024

facebook-github-bot commented Mar 19, 2024

facebook-github-bot commented Mar 19, 2024

mbasmanova commented Mar 22, 2024

Add support for DECIMAL types to Simple Function API #9096

Add support for DECIMAL types to Simple Function API #9096

Conversation

mbasmanova commented Mar 15, 2024 • edited Loading

facebook-github-bot commented Mar 15, 2024

netlify bot commented Mar 15, 2024 • edited Loading

✅ Deploy Preview for meta-velox canceled.

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 15, 2024

facebook-github-bot commented Mar 18, 2024

facebook-github-bot commented Mar 19, 2024

facebook-github-bot commented Mar 19, 2024

facebook-github-bot commented Mar 19, 2024

facebook-github-bot commented Mar 19, 2024

mbasmanova commented Mar 22, 2024

mbasmanova commented Mar 15, 2024 •

edited

Loading

netlify bot commented Mar 15, 2024 •

edited

Loading