Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: user UUIDs on export for Native Filter Configuration #18562

Merged
merged 10 commits into from
Feb 8, 2022

Conversation

hughhhh
Copy link
Member

@hughhhh hughhhh commented Feb 3, 2022

SUMMARY

Updating the current process of exporting dashboard native filter configurations to store uuids vs chartids. The problem is when users export charts from 1 superset instance (workspace) to another the chart id reference aren't the same. So fix that we are allowing the native filters to leverage uuids from the charts and then updating them once they are properly loaded into the workspace/instance.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@pull-request-size pull-request-size bot added size/S and removed size/XL labels Feb 3, 2022
@codecov
Copy link

codecov bot commented Feb 3, 2022

Codecov Report

Merging #18562 (ad0552d) into master (f7f1df2) will increase coverage by 0.00%.
The diff coverage is 77.77%.

❗ Current head ad0552d differs from pull request most recent head 85634b7. Consider uploading reports for the commit 85634b7 to get more accurate results

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #18562   +/-   ##
=======================================
  Coverage   66.29%   66.29%           
=======================================
  Files        1594     1594           
  Lines       62623    62626    +3     
  Branches     6312     6312           
=======================================
+ Hits        41518    41521    +3     
  Misses      19456    19456           
  Partials     1649     1649           
Flag Coverage Δ
hive 52.06% <25.00%> (-0.07%) ⬇️
mysql 81.25% <100.00%> (+<0.01%) ⬆️
postgres 81.30% <100.00%> (+<0.01%) ⬆️
presto 51.91% <25.00%> (-0.07%) ⬇️
python 81.73% <100.00%> (+<0.01%) ⬆️
sqlite 80.99% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...tend/src/SqlLab/components/ColumnElement/index.tsx 93.75% <ø> (ø)
...frontend/src/SqlLab/components/ResultSet/index.tsx 50.73% <ø> (ø)
...rset-frontend/src/components/ListView/ListView.tsx 96.25% <ø> (ø)
.../explore/components/ExploreViewContainer/index.jsx 57.22% <0.00%> (ø)
...ols/DndColumnSelectControl/ColumnSelectPopover.tsx 3.89% <0.00%> (ø)
.../controls/DndColumnSelectControl/OptionWrapper.tsx 63.41% <ø> (ø)
superset/charts/api.py 85.93% <ø> (ø)
superset/charts/schemas.py 99.34% <ø> (ø)
superset/dashboards/api.py 92.54% <ø> (ø)
superset/databases/api.py 93.99% <ø> (ø)
... and 26 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f7f1df2...85634b7. Read the comment docs.

Copy link
Member

@betodealmeida betodealmeida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hughhhh I think instead of changing the export, it's better to simply fix the import to update excluded.

if you change the export so that native_filter["scope"]["excluded"] contains UUIDs, not IDs, you need to add logic to the import so it can handle both cases (UUIDs vs. IDs in exluded), in order to maintain backward compatibility.

During dashboard import we already compute a map between the old IDs and the new ones:

# build map old_id => new_id
old_ids = build_uuid_to_id_map(fixed["position"])

That function, update_id_refs, is already responsible for updating the dashboard metadata to use the new IDs after an import. I think the best solution here is to modify it to also update native_filter["scope"]["excluded"].

@pull-request-size pull-request-size bot added size/S and removed size/M labels Feb 7, 2022
@@ -17,7 +17,7 @@

import json
import logging
from typing import Any, Dict, Set
from typing import Any, Callable, Dict, Set, Union
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new imports are not needed, right?

@@ -49,8 +49,10 @@ def test_update_id_refs_immune_missing( # pylint: disable=invalid-name
"101": {"filter_name": {"immune": [102, 103]}},
"104": {"filter_name": {"immune": [102, 103]}},
},
"native_filter_configuration": [
{"scope": {"excluded": ["uuid1", "uuid2"]}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be IDs, not UUIDs.

Also, can we have a new test to check the logic, instead of piggybacking on test_update_id_refs_immune_missing?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rookie mistake this is on import i will update accordingly 😅

@pull-request-size pull-request-size bot added size/M and removed size/S labels Feb 7, 2022
Copy link
Member

@betodealmeida betodealmeida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks for fixing this, @hughhhh!

superset/dashboards/commands/importers/v1/utils.py Outdated Show resolved Hide resolved
hughhhh and others added 2 commits February 8, 2022 10:53
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
@hughhhh hughhhh merged commit 7194a01 into master Feb 8, 2022
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
@mistercrunch mistercrunch deleted the hugh-save-import branch March 26, 2024 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/M 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants