Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collate Aware Postgresql String Builder #2216

Merged
merged 1 commit into from
Sep 5, 2024
Merged

Conversation

AbdulR3hman
Copy link
Contributor

Issue #, if available:

Description of changes:
Postgresql default collate setting differs from Athena's and thus causes issues that results of missing records; for example: PostgreSQL 'a' < 'A' & Trino 'A' < 'a'. This causes the dynamic filtering range to miss records.

This change will allow customers to enforce Collate "C" to any varchar based predicate.

warning: this is only an experimental feature.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@@ -349,11 +349,4 @@ else if (singleValues.size() > 1) {

return "(" + Joiner.on(" OR ").join(disjuncts) + ")";
}

private String toPredicate(String columnName, String operator, Object value, ArrowType type,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the exact method that was in parent; and since I made parent protected; and this is exact duplicate; there was no reason to keep it.

@burhan94
Copy link
Contributor

burhan94 commented Sep 3, 2024

Can we add a test case to the integ test package?

ty

@AbdulR3hman AbdulR3hman merged commit 856418b into master Sep 5, 2024
8 checks passed
@AbdulR3hman AbdulR3hman deleted the postgresql-collate branch September 5, 2024 17:49
github-actions bot pushed a commit that referenced this pull request Sep 11, 2024
  - Add athena connector exception class to classify Runtime Exception (#2241)
  - build(deps): bump software.amazon.awssdk:bom from 2.27.17 to 2.27.21 (#2249)
  - build(deps): bump software.amazon.awssdk:bom from 2.27.17 to 2.27.21
  - build(deps-dev): bump log4j2Version from 2.23.1 to 2.24.0 (#2247)
  - build(deps-dev): bump log4j2Version from 2.23.1 to 2.24.0
  - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client from 8.15.0 to 8.15.1 (#2246)
  - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client
  - build(deps): bump aws-sdk.version from 1.12.770 to 1.12.771 (#2245)
  - build(deps): bump aws-sdk.version from 1.12.770 to 1.12.771
  - Updated Postgresql Split Query Comment (#2242)
  - Collate Aware Postgresql String Builder (#2216)
  - build(deps): bump org.yaml:snakeyaml from 2.2 to 2.3 (#2236)
  - build(deps): bump org.yaml:snakeyaml from 2.2 to 2.3
  - build(deps): bump surefire.failsafe.version from 3.4.0 to 3.5.0 (#2233)
  - build(deps): bump surefire.failsafe.version from 3.4.0 to 3.5.0
  - build(deps): bump net.snowflake:snowflake-jdbc from 3.18.0 to 3.19.0 (#2235)
  - build(deps): bump net.snowflake:snowflake-jdbc from 3.18.0 to 3.19.0
  - build(deps): bump org.apache.maven.plugins:maven-javadoc-plugin from 3.8.0 to 3.10.0 (#2234)
  - build(deps): bump org.apache.maven.plugins:maven-javadoc-plugin
  - build(deps): bump com.microsoft.azure:msal4j from 1.17.0 to 1.17.1 (#2231)
  - build(deps): bump com.microsoft.azure:msal4j from 1.17.0 to 1.17.1
  - build(deps): bump org.apache.commons:commons-lang3 from 3.16.0 to 3.17.0 (#2237)
  - build(deps): bump org.apache.commons:commons-lang3 from 3.16.0 to 3.17.0
  - build(deps): bump com.clickhouse:clickhouse-jdbc from 0.6.4 to 0.6.5 (#2232)
  - build(deps): bump com.clickhouse:clickhouse-jdbc from 0.6.4 to 0.6.5
  - build(deps): bump software.amazon.jsii:jsii-runtime from 1.102.0 to 1.103.1 (#2229)
  - build(deps): bump software.amazon.jsii:jsii-runtime
  - build(deps): bump software.amazon.awssdk:bom from 2.27.12 to 2.27.17 (#2230)
  - build(deps): bump software.amazon.awssdk:bom from 2.27.12 to 2.27.17
  - Don't declare jsii-runtime as a dependency of any component directly (#2228)
  - build(deps): bump software.amazon.awssdk:bom from 2.27.10 to 2.27.12 (#2214)
  - build(deps): bump software.amazon.awssdk:bom from 2.27.10 to 2.27.12
  - build(deps-dev): bump nl.jqno.equalsverifier:equalsverifier from 3.16.1 to 3.16.2 (#2213)
  - build(deps-dev): bump nl.jqno.equalsverifier:equalsverifier
  - build(deps): bump com.microsoft.sqlserver:mssql-jdbc from 12.8.0.jre11 to 12.8.1.jre11 (#2212)
  - build(deps): bump com.microsoft.sqlserver:mssql-jdbc
  - build(deps): bump com.microsoft.azure:msal4j from 1.16.2 to 1.17.0 (#2210)
  - build(deps): bump com.microsoft.azure:msal4j from 1.16.2 to 1.17.0
  - build(deps): bump org.apache.maven.plugins:maven-checkstyle-plugin from 3.4.0 to 3.5.0 (#2211)
  - build(deps): bump org.apache.maven.plugins:maven-checkstyle-plugin
  - build(deps): bump org.postgresql:postgresql from 42.7.3 to 42.7.4 (#2209)
  - build(deps): bump org.postgresql:postgresql from 42.7.3 to 42.7.4
  - Revert "build(deps): bump software.amazon.msk:aws-msk-iam-auth from 2.1.1 to 2.2.0" (#2181)
  - build(deps): bump com.google.cloud:google-cloud-storage from 2.41.0 to 2.42.0 (#2189)
  - build(deps): bump com.google.cloud:google-cloud-storage
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants