Billing API IsBillingEnabled #626
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## billing-release-002 #626 +/- ##
=======================================================
+ Coverage 69.59% 69.66% +0.07%
=======================================================
Files 117 117
Lines 9567 9561 -6
=======================================================
+ Hits 6658 6661 +3
+ Misses 2909 2900 -9 ☔ View full report in Codecov by Sentry. |
violetbrina
approved these changes
Nov 27, 2023
Yes, I did test it, both with SM_GCP_BQ_AGGREG_VIEW defined and undefined. |
milo-hyben
added a commit
that referenced
this pull request
Jan 25, 2024
* Billing api extra labels (#619) * Added compute_category, cromwell_sub_workflow_name, cromwell_workflow_id, goog_pipelines_worker and wdl_task_name to extended view and created relevant filters and API points. * Added labels to all BQ queries, refactoring billing layer. * Added examples to billing-total-cost API regarding the new filters. * Billing - fixing styling issues after the first Billing release (#624) * Temporarily disable seqr and hail from /topics API. * Autoselect 1st topic / 1st project value from the DDL. * Merging Billing.css into index.css * Small fix - reusing extRecords in FieldSelector component. * Refactoring duplicated code in FieldSelector. * Added Stages to the Group by DDL. * Billing API IsBillingEnabled (#626) * Added API point to check if billing is enabled. * Added simple Total Cost By Batch Page. (#627) * Added simple Total Cost By Batch Page. * Billing cost by category (#629) * Added simple Total Cost By Batch Page. * Fixed autoselect day format. * Fixing day format for autoselect (missing leading 0) * Added first draft of billing page to show detail SKU per selected cost category over selected time periods (day, week, month or invoice month) * Small fix for BillingCostByBatch page, disable search if searchBy is empty or < 6 chars. * New: Billing API GET namespaces, added namespace to allowed fields for total cost. * Implemented HorizontalStackedBarChart, updated Billing By Invoice Month page to enable toggle between chart and table view. * Stacked Bars Chart with option to accumulate data. (#634) * Implemented Stacked bars with option to accumulate data. * Added budget bar to billing horizontal bar chart, added background color for the billing table to reflect the chart colours. * Added simple prediction of billing stacked bar chart. * Billing hail batch layout (#633) * Added simple Total Cost By Batch Page. * Removing debug prints. * Fixed autoselect day format. * Fixing day format for autoselect (missing leading 0) * Added first draft of billing page to show detail SKU per selected cost category over selected time periods (day, week, month or invoice month) * Small fix for BillingCostByBatch page, disable search if searchBy is empty or < 6 chars. * New: Billing API GET namespaces, added namespace to allowed fields for total cost. * Implemented HorizontalStackedBarChart, updated Billing By Invoice Month page to enable toggle between chart and table view. * ADD: Cost by Analysis page * ADD: add start of Analysis grid * ADD: add start of Analysis grid * FIX: table fixes for the HailBatchGrid * API: api changes to enable query of the raw table * API: fixed and working with updated get_total_cost endpoint * API: fix typing of get_total_cost (default return is now a list[dict] and can be converted in the layer/route to a specific output type * API: add endpoint to get costs by batch_id * API: done * IN PROGRESS: modifying Cost By Analysis to use new endpoints * IN PROGRESS: changes to Cost By Analysis, linking with backend API. * IN PROGRESS: changes to Cost By Analysis, grid grouping by ar/batch/job. * NEW: finalising Cost By Analysis page * ADD: durations to Cost By Analysis page --------- Co-authored-by: Milo Hyben <milo.hyben@gmail.com> * FIX: Billing - fixing time_column condition. * Removing draft billing page. * Remove unused API point & cleanup, changes as per code review. * Small Frontend refactoring, reflecting PR review. * Updating billing style for dark mode. * Optimised Frontend, replacing reduce with forEach where possible. * Refactoring Billing DB structures. * Cleaning up unused dependencies. * FIX: replaced button 'color=red' with 'negative' property. * FIX: replace HEX color for pattern with CSS var. * FIX: replace async call with sync for a simple function. * FIX: dark mode for Horizontal Stacked Bar. * FIX: billing cost by analysis page, esp. search control resizing and functionality. * FIX: duplicated keys in the grid on Billing Cost By Analysis page. * FIX: refactoring BQ tables, small fixes for billing pages. * FIX: BillingCostPageAnalysis, keeping the old record until loading of data finishes. * FIX: Billing StackedChart various issues. * FIX: missing filters checks, updating charts when loading. * FIX: silenece linting no attribute msg for Middleware. * Refactoring filters, implemented first Billing GraphQL integration. * Removing temporary Billing GraphQL, will be properly implemented in the next PRs. * Applied changes as suggested by review. * Update db/python/tables/bq/generic_bq_filter_model.py Co-authored-by: Michael Franklin <22381693+illusional@users.noreply.github.com> --------- Co-authored-by: Sabrina Yan <9669990+violetbrina@users.noreply.github.com> Co-authored-by: Michael Franklin <22381693+illusional@users.noreply.github.com>
Merged
milo-hyben
added a commit
that referenced
this pull request
Feb 22, 2024
* Billing api extra labels (#619) * Added compute_category, cromwell_sub_workflow_name, cromwell_workflow_id, goog_pipelines_worker and wdl_task_name to extended view and created relevant filters and API points. * Added labels to all BQ queries, refactoring billing layer. * Added examples to billing-total-cost API regarding the new filters. * Billing - fixing styling issues after the first Billing release (#624) * Temporarily disable seqr and hail from /topics API. * Autoselect 1st topic / 1st project value from the DDL. * Merging Billing.css into index.css * Small fix - reusing extRecords in FieldSelector component. * Refactoring duplicated code in FieldSelector. * Added Stages to the Group by DDL. * Billing API IsBillingEnabled (#626) * Added API point to check if billing is enabled. * Added simple Total Cost By Batch Page. (#627) * Added simple Total Cost By Batch Page. * Billing cost by category (#629) * Added simple Total Cost By Batch Page. * Fixed autoselect day format. * Fixing day format for autoselect (missing leading 0) * Added first draft of billing page to show detail SKU per selected cost category over selected time periods (day, week, month or invoice month) * Small fix for BillingCostByBatch page, disable search if searchBy is empty or < 6 chars. * New: Billing API GET namespaces, added namespace to allowed fields for total cost. * Implemented HorizontalStackedBarChart, updated Billing By Invoice Month page to enable toggle between chart and table view. * Stacked Bars Chart with option to accumulate data. (#634) * Implemented Stacked bars with option to accumulate data. * Added budget bar to billing horizontal bar chart, added background color for the billing table to reflect the chart colours. * Added simple prediction of billing stacked bar chart. * Billing hail batch layout (#633) * Added simple Total Cost By Batch Page. * Removing debug prints. * Fixed autoselect day format. * Fixing day format for autoselect (missing leading 0) * Added first draft of billing page to show detail SKU per selected cost category over selected time periods (day, week, month or invoice month) * Small fix for BillingCostByBatch page, disable search if searchBy is empty or < 6 chars. * New: Billing API GET namespaces, added namespace to allowed fields for total cost. * Implemented HorizontalStackedBarChart, updated Billing By Invoice Month page to enable toggle between chart and table view. * ADD: Cost by Analysis page * ADD: add start of Analysis grid * ADD: add start of Analysis grid * FIX: table fixes for the HailBatchGrid * API: api changes to enable query of the raw table * API: fixed and working with updated get_total_cost endpoint * API: fix typing of get_total_cost (default return is now a list[dict] and can be converted in the layer/route to a specific output type * API: add endpoint to get costs by batch_id * API: done * IN PROGRESS: modifying Cost By Analysis to use new endpoints * IN PROGRESS: changes to Cost By Analysis, linking with backend API. * IN PROGRESS: changes to Cost By Analysis, grid grouping by ar/batch/job. * NEW: finalising Cost By Analysis page * ADD: durations to Cost By Analysis page --------- Co-authored-by: Milo Hyben <milo.hyben@gmail.com> * FIX: Billing - fixing time_column condition. * Removing draft billing page. * Remove unused API point & cleanup, changes as per code review. * Small Frontend refactoring, reflecting PR review. * Updating billing style for dark mode. * Optimised Frontend, replacing reduce with forEach where possible. * Refactoring Billing DB structures. * Cleaning up unused dependencies. * FIX: replaced button 'color=red' with 'negative' property. * FIX: replace HEX color for pattern with CSS var. * FIX: replace async call with sync for a simple function. * FIX: dark mode for Horizontal Stacked Bar. * FIX: billing cost by analysis page, esp. search control resizing and functionality. * FIX: duplicated keys in the grid on Billing Cost By Analysis page. * FIX: refactoring BQ tables, small fixes for billing pages. * FIX: BillingCostPageAnalysis, keeping the old record until loading of data finishes. * FIX: Billing StackedChart various issues. * Linting * FIX: missing filters checks, updating charts when loading. * FIX: silenece linting no attribute msg for Middleware. * Refactoring filters, implemented first Billing GraphQL integration. * Fixing linting. * Added unit tests for BQ filters. * Fixing linting. * Added tests for billing routes. * Removing billing GraphQL, there will be another PR for this. * Adding doc strings to tests. * Changing to staticmethod where relevant, added more unitests. * Linting * Refactoring string constant so both pylint and unittest are happy. * Unittests for BQ Function filter. * Unittests for BillingArBatchTable. * Added pytz dependency to dev so we can write unit test with timezone aware datetime. * Fixing timezone aware unit test for billing function filter. * Refactoring BQ Unittests, added more for BillingBaseTable class. * Linting * Added more unit tests for BillingBaseTable. * Add last missing unittest for BilingBaseTable. * Added unittests for BillingLayer. * Linting. * Merge dev try 2. * More unit tests for BillingLayer. * More unit tests for Billing routes functions. * More unit tests for Billing APi routes. * Billing unit tests - use mockup author. * Added BillingLayer muckup to unit tests. * Billing unit tests refactoring, implementing feedback from PR. * Fixing billing unit tests. * More fixes for billing unit tests. * Removing tests related to old getLabelValue functions. * Update test/test_api_billing.py Co-authored-by: Michael Franklin <22381693+illusional@users.noreply.github.com> * Setting up mock of is_billing_enabled in test class setUp function. --------- Co-authored-by: Sabrina Yan <9669990+violetbrina@users.noreply.github.com> Co-authored-by: Michael Franklin <22381693+illusional@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added simple API point to check if Billing is enabled.
ATM it checks only for env var regarding Billing Aggreg view, which should be enough.
I do not think we would need to check all the billing env variables.
I had added as well check to each billing API point, so it fails soon.