Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Billing pre-release #605

Merged
merged 18 commits into from
Nov 19, 2023
Merged

Billing pre-release #605

merged 18 commits into from
Nov 19, 2023

Conversation

milo-hyben
Copy link
Contributor

This PR contains all the changes before release.

violetbrina and others added 13 commits October 23, 2023 16:18
* START: add billing home page and data page

* REFACTOR: NavBar

* REFACTOR: NavBar

* REFACTOR: NavBar - adding dropdown meny capability

* ../

* FIX: fixed style in the nav. Everythign but search bar

* FIX: search bar fixed

* DONE: menu and billing pages done

* REMOVE: remove redundant code

* LINT: fix format
* User Story 1.3: As a user, I want to sort costs by budget and by the percentage of budget used

* First version of Current Billing Cost page.

* Added dropdown to select grouping by Project, Topic or Dataset.
…ries with daily granularity in the billing dashboard (#588)

* Added gcp-projects API to billing, new Billing CostByTime page

* Updated StackedAreaByDateChart wiht more custom properties.
* FIX: navbar style fix

* FIX: Dropdown to FieldSelector in BillingInvoiceMonthCost

* API: add api changes to allow the get running cost query to filter using invoice month

* API: add api changes to allow the get running cost query to filter using invoice month

* FIX: Dropdown to FieldSelector in BillingInvoiceMonthCost

* FIX: all pages and navigation/links working

* FIX: fixed all of the navigate() issues now all links work

* IN PROGRESS: trying to cache the call to the seqr prop map API endpoint. UI cleaned up.

* LINT: fix typing issues in API

* RM: console.log
* REFACOR: move data loading logic into the main page to be used by all charts

* LINT: fix format

* FIX: bug fix for _.uniq issue

* LINT: fix format
* IN PROGRESS: data table on over time cost page

* DONE: data table on time view page complete

* LINT: fix format
* Added gcp-projects API to billing, new Billing CostByTime page

* Updated StackedAreaByDateChart with more custom properties.

* First version of Bar and Donut charts.

* Upgrading babel / vulnerability.
* Bill 112 (#583)

* START: add billing home page and data page

* REFACTOR: NavBar

* REFACTOR: NavBar

* REFACTOR: NavBar - adding dropdown meny capability

* ../

* FIX: fixed style in the nav. Everythign but search bar

* FIX: search bar fixed

* DONE: menu and billing pages done

* REMOVE: remove redundant code

* LINT: fix format

* [BIL-26] BILLING - User Story 1.3 (#578)

* User Story 1.3: As a user, I want to sort costs by budget and by the percentage of budget used

* First version of Current Billing Cost page.

* Added dropdown to select grouping by Project, Topic or Dataset.

* [BIL-39] User Story 2.3: As a user, I want to view costs in a time series with daily granularity in the billing dashboard (#588)

* Added gcp-projects API to billing, new Billing CostByTime page

* Updated StackedAreaByDateChart wiht more custom properties.

* Bill 150 (#589)

* FIX: navbar style fix

* FIX: Dropdown to FieldSelector in BillingInvoiceMonthCost

* API: add api changes to allow the get running cost query to filter using invoice month

* API: add api changes to allow the get running cost query to filter using invoice month

* FIX: Dropdown to FieldSelector in BillingInvoiceMonthCost

* FIX: all pages and navigation/links working

* FIX: fixed all of the navigate() issues now all links work

* IN PROGRESS: trying to cache the call to the seqr prop map API endpoint. UI cleaned up.

* LINT: fix typing issues in API

* RM: console.log

* Extended bq looks back time to 300 days as we do not have much loaded in the dev table. (#591)

* Billing cost by time data refactor (#592)

* REFACOR: move data loading logic into the main page to be used by all charts

* LINT: fix format

* FIX: bug fix for _.uniq issue

* LINT: fix format

* Bil 225 (#596)

* IN PROGRESS: data table on over time cost page

* DONE: data table on time view page complete

* LINT: fix format

* FIX: rename of variables

* UPDATE: frontend now checks if the the billing endpoint is returning an OK status or not. Hides billing pages on any failed (not 200) status

---------

Co-authored-by: Milo Hyben <milo-hyben@users.noreply.github.com>
* Added gcp-projects API to billing, new Billing CostByTime page

* Updated StackedAreaByDateChart with more custom properties.

* First version of Bar and Dount charts.

* Upgrading babel / vulnerability.
* Fixing last 24H billing calculations.

* Limit Budget % and 24H to only latest month.

* Pick only the latest monthly budget row per gcp_project.

* Added Last 24H UTC day to the table header.

* Removing unused job_config.
@milo-hyben milo-hyben changed the title Billing Billing pre-release Nov 10, 2023
@codecov-commenter
Copy link

codecov-commenter commented Nov 12, 2023

Codecov Report

Attention: 146 lines in your changes are missing coverage. Please review.

Comparison is base (6d01ce6) 70.56% compared to head (c3722cd) 69.66%.
Report is 2 commits behind head on dev.

Files Patch % Lines
db/python/layers/billing.py 14.17% 115 Missing ⚠️
api/utils/dates.py 22.22% 14 Missing ⚠️
api/routes/billing.py 42.85% 12 Missing ⚠️
models/models/billing.py 84.37% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #605      +/-   ##
==========================================
- Coverage   70.56%   69.66%   -0.91%     
==========================================
  Files         116      116              
  Lines        9316     9517     +201     
==========================================
+ Hits         6574     6630      +56     
- Misses       2742     2887     +145     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@illusional illusional left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pass 1, I haven't had an opportunity to look a the front-end changes yet.

api/routes/__init__.py Outdated Show resolved Hide resolved
api/routes/analysis.py Outdated Show resolved Hide resolved
api/routes/billing.py Outdated Show resolved Hide resolved
db/python/layers/analysis.py Outdated Show resolved Hide resolved
db/python/layers/billing.py Outdated Show resolved Hide resolved
db/python/layers/billing.py Outdated Show resolved Hide resolved
models/models/billing.py Outdated Show resolved Hide resolved
web/src/Routes.tsx Show resolved Hide resolved
web/src/pages/billing/Billing.css Show resolved Hide resolved
web/src/pages/billing/BillingCostByTime.tsx Outdated Show resolved Hide resolved
milo-hyben and others added 4 commits November 14, 2023 09:18
* FIX: revert analysi.py, fix unused import

* FIX: endpoint naming consistency

* FIX: additional get_invoice_month_range comment detail

* FIX: revert analysis.py db layer

* FIX: remove hard coding in billing.py db layer

* FIX: add clarrifying comments to billing.py layer

* FIX: switch to defaultdict in billing layer, also some error message clarrification

* FIX: Decimal to float
@violetbrina violetbrina marked this pull request as ready for review November 15, 2023 23:30
@milo-hyben milo-hyben merged commit bdb637d into dev Nov 19, 2023
2 checks passed
@milo-hyben milo-hyben deleted the billing branch November 19, 2023 23:02
milo-hyben added a commit that referenced this pull request Nov 20, 2023
* Bill 112 (#583)

* START: add billing home page and data page

* FIX: fixed style in the nav. Everythign but search bar

* DONE: menu and billing pages done

* REMOVE: remove redundant code

* User Story 1.3: As a user, I want to sort costs by budget and by the percentage of budget used

* First version of Current Billing Cost page.

* Added dropdown to select grouping by Project, Topic or Dataset.

* [BIL-39] User Story 2.3: As a user, I want to view costs in a time series with daily granularity in the billing dashboard (#588)

* Added gcp-projects API to billing, new Billing CostByTime page

* Updated StackedAreaByDateChart wiht more custom properties.

* Bill 150 (#589)

* API: add api changes to allow the get running cost query to filter using invoice month

* IN PROGRESS: trying to cache the call to the seqr prop map API endpoint. UI cleaned up.

* RM: console.log

* Extended bq looks back time to 300 days as we do not have much loaded in the dev table. (#591)

* REFACOR: move data loading logic into the main page to be used by all charts

* DONE: data table on time view page complete

* Added gcp-projects API to billing, new Billing CostByTime page

* Updated StackedAreaByDateChart with more custom properties.

* First version of Bar and Donut charts.

* Upgrading babel / vulnerability.

* Bil 242 - Hide billing pages when env variables aren't set (#600)

* DONE: menu and billing pages done

* REMOVE: remove redundant code

* API: add api changes to allow the get running cost query to filter using invoice month

* FIX: Dropdown to FieldSelector in BillingInvoiceMonthCost

* FIX: all pages and navigation/links working

* FIX: fixed all of the navigate() issues now all links work

* IN PROGRESS: trying to cache the call to the seqr prop map API endpoint. UI cleaned up.

* LINT: fix typing issues in API

* RM: console.log

* Extended bq looks back time to 300 days as we do not have much loaded in the dev table. (#591)

* Billing cost by time data refactor (#592)

* IN PROGRESS: data table on over time cost page

* DONE: data table on time view page complete

* UPDATE: frontend now checks if the the billing endpoint is returning an OK status or not. Hides billing pages on any failed (not 200) status

---------



* Added gcp-projects API to billing, new Billing CostByTime page

* Updated StackedAreaByDateChart with more custom properties.

* First version of Bar and Dount charts.

* Upgrading babel / vulnerability.

* Billing - show 24H table fields only to the current invoice month (#604)

* Fixing last 24H billing calculations.

* Limit Budget % and 24H to only latest month.

* Pick only the latest monthly budget row per gcp_project.

* Added Last 24H UTC day to the table header.

* Removing unused job_config.

* Fixing docker image building issues.

* Getting gcp_project data from gcp_billing view instead of aggregated view. (#606)

* Small fix to enable query cost by ar-guid, added example to API docs. (#611)

---------

Co-authored-by: Sabrina Yan <9669990+violetbrina@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants