Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bigquery): implement custom minute time grains #12581

Merged
merged 2 commits into from
Jan 19, 2021

Conversation

villebro
Copy link
Member

SUMMARY

Currently the BigQuery engine spec only supports full seconds/minutes/hours due to the dialect not natively supporting custom intervals. This PR adds support for the missing default 5, 10, 15 and 30 min grains by utilizing some custom timestamp handling functions:

Screenshot

image

TEST PLAN

Verified to work on TIMESTAMP, DATETIME and DATE types + added new unit tests.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Requires DB Migration.
  • Confirm DB Migration upgrade and downgrade tested.
  • Introduces new feature or API
  • Removes existing feature or API

actual = BigQueryEngineSpec.get_timestamp_expr(
col=col, pdf=None, time_grain="PT5M", type_=type_
)
self.assertEqual(str(actual), expected)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we are using assert instead of self.assertEqual

@codecov-io
Copy link

codecov-io commented Jan 18, 2021

Codecov Report

Merging #12581 (25a951f) into master (0f243c6) will decrease coverage by 2.77%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #12581      +/-   ##
==========================================
- Coverage   66.75%   63.97%   -2.78%     
==========================================
  Files        1015      486     -529     
  Lines       49634    29970   -19664     
  Branches     4839        0    -4839     
==========================================
- Hits        33131    19173   -13958     
+ Misses      16380    10797    -5583     
+ Partials      123        0     -123     
Flag Coverage Δ
cypress ?
javascript ?
python 63.97% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/db_engine_specs/bigquery.py 95.74% <ø> (ø)
superset/db_engine_specs/base.py 85.96% <100.00%> (+0.14%) ⬆️
superset-frontend/src/logger/LogUtils.ts
...perset-frontend/src/common/components/Dropdown.tsx
...src/SqlLab/utils/reduxStateToLocalStorageHelper.js
...src/components/FilterableTable/FilterableTable.tsx
...-frontend/src/components/Button/Button.stories.jsx
...rontend/src/visualizations/TimeTable/TimeTable.jsx
.../src/explore/components/MetricDefinitionOption.jsx
...ard/components/filterscope/FilterScopeSelector.jsx
... and 522 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f243c6...25a951f. Read the comment docs.

@adam-stasiak
Copy link
Contributor

Tested manually - ok :) I had only problem with connection to physical table (not from sql editor). But time columns mentioned in description worked. 🟢

@villebro
Copy link
Member Author

@adam-stasiak good to know - can you open a ticket about that? I'm pretty sure it's unrelated, but needs to be fixed at any rate but in a different PR.

@villebro villebro merged commit 7e99d76 into apache:master Jan 19, 2021
@villebro villebro deleted the villebro/bq-custom-grain branch January 19, 2021 12:32
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.2.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels preset-io size/M 🚢 1.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants