Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(config): Add none force-refresh option for auto refresh #17039

Merged
merged 4 commits into from
Oct 14, 2021

Conversation

yongchand
Copy link
Contributor

@yongchand yongchand commented Oct 8, 2021

SUMMARY

Add alternative option for issue #16944
If people think this should be default, I may just remove from config and simply change force -> fetch

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Before:
https://user-images.githubusercontent.com/43336767/136601879-c3ea7040-5944-43b6-bf05-15d7f29ef22a.mov

When DASHBOARD_FORCE_AUTO_REFRESH is True, chart will query every time when it auto-refreshes.

After:
https://user-images.githubusercontent.com/43336767/136601893-a876f406-6190-4d90-bffb-e78e4eaa8661.mov

When DASHBOARD_FORCE_AUTO_REFRESH is False, chart will use cache if it exist during auto-refresh.

TESTING INSTRUCTIONS

Since this PR requires to change superset-ui to change, you can test by following step instead.

  1. Open dashboard with several charts with cache of certain period (Say 300s)
  2. Check that chart will not use cache at all (like before video) during auto-refresh
  3. Change fetchCharts(xx, True, xx, xx) to fetchCharts(xx, false, xx, xx)
  4. Check that chart use cache during auto-refresh

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@yongchand yongchand changed the title feat(config): Add none force-refresh option for auto refresh [WIP] feat(config): Add none force-refresh option for auto refresh Oct 8, 2021
@yongchand
Copy link
Contributor Author

Thanks @villebro, I just added PR :) I think it is better to add as a config option so users can choose - what do you think?

@codecov
Copy link

codecov bot commented Oct 11, 2021

Codecov Report

Merging #17039 (27afd22) into master (2c8e06e) will decrease coverage by 0.26%.
The diff coverage is 60.97%.

❗ Current head 27afd22 differs from pull request most recent head 38c1a97. Consider uploading reports for the commit 38c1a97 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17039      +/-   ##
==========================================
- Coverage   76.92%   76.66%   -0.27%     
==========================================
  Files        1031     1031              
  Lines       55157    55200      +43     
  Branches     7501     7507       +6     
==========================================
- Hits        42430    42318     -112     
- Misses      12475    12630     +155     
  Partials      252      252              
Flag Coverage Δ
hive ?
javascript 70.79% <54.92%> (-0.09%) ⬇️
presto ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...frontend/src/dashboard/components/Header/index.jsx 68.71% <0.00%> (-0.78%) ⬇️
...frontend/src/profile/components/CreatedContent.tsx 87.50% <ø> (ø)
.../database/DatabaseModal/DatabaseConnectionForm.tsx 48.71% <ø> (ø)
...ontend/src/views/CRUD/data/dataset/DatasetList.tsx 66.29% <0.00%> (+0.37%) ⬆️
superset/db_engine_specs/ascend.py 100.00% <ø> (ø)
superset/db_engine_specs/athena.py 89.28% <ø> (ø)
superset/db_engine_specs/base.py 88.17% <ø> (-0.39%) ⬇️
superset/db_engine_specs/bigquery.py 85.97% <ø> (ø)
superset/db_engine_specs/clickhouse.py 66.03% <ø> (ø)
superset/db_engine_specs/crate.py 92.30% <ø> (ø)
... and 46 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c8e06e...38c1a97. Read the comment docs.

@geido
Copy link
Member

geido commented Oct 11, 2021

Hello @yongchand thanks for the contribution!

I am thinking that feature flags might not be the best place for this type of setting. @villebro correct me if I am wrong, but being a frontend-only constant, probably superset-frontend/src/constants.ts could be a better place for this one.

@yongchand
Copy link
Contributor Author

yongchand commented Oct 11, 2021

@geido Thanks for your opinion!
I was thinking that user should choose whether to enable force-refresh (for auto refresh).
Therefore, config.py could be a good place to simply change the configuration.
In config.py, feature flag like ENABLE_DND_WITH_CLICK_UX is only used on frontend-only too as long as I know.
Please let me know if I am misunderstanding!

@villebro
Copy link
Member

villebro commented Oct 11, 2021

I agree with @geido that this should be a general config, not a feature flag. I would propose adding something similar to this to config.py:

DASHBOARD_AUTO_REFRESH_MODE: Literal["fetch", "force"] = "force"

Then that needs to be added to bootstrap data by adding the key to FRONTEND_CONF_KEYS on the backend. After this the value should be available on the dashboard on dashboardInfo.common.conf. But I may be oversimplifying things - let me know if you need help @yongchand!

@yongchand yongchand changed the title [WIP] feat(config): Add none force-refresh option for auto refresh feat(config): Add none force-refresh option for auto refresh Oct 11, 2021
@yongchand
Copy link
Contributor Author

yongchand commented Oct 11, 2021

Tested locally :) Works totally fine! @geido @villebro
Feedback welcomed :)

@villebro
Copy link
Member

Thanks @yongchand! I started CI and will test today

@yongchand
Copy link
Contributor Author

yongchand commented Oct 13, 2021

Done with lint issues! All fixed! (I hope) :)
@villebro

@villebro
Copy link
Member

Thanks @yongchand; CI started 👍

superset/config.py Outdated Show resolved Hide resolved
@yongchand
Copy link
Contributor Author

yongchand commented Oct 13, 2021

@villebro sorry for keep pinging you! Just fixed isort issue. If this goes fine, please merge whenever you are available :)

@villebro
Copy link
Member

sorry for keep pinging you! Just fixed isort issue. If this goes fine, please merge whenever you are available :)

No worries @yongchand, CI restarted 🙂

@yongchand
Copy link
Contributor Author

sorry for keep pinging you! Just fixed isort issue. If this goes fine, please merge whenever you are available :)

No worries @yongchand, CI restarted 🙂

Everything seems fine except for cypress-matrix (which seemed to be fine on last run). Do you possibly know the reason? @villebro

@villebro
Copy link
Member

Everything seems fine except for cypress-matrix (which seemed to be fine on last run). Do you possibly know the reason?

@yongchand seems to be a flaky test - I just restarted it, will try to monitor it today to make sure it passes

@yongchand
Copy link
Contributor Author

yongchand commented Oct 14, 2021

Seems good now! Thanks :) @villebro

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for all the iterations @yongchand !

@villebro villebro merged commit 7a2e394 into apache:master Oct 14, 2021
opus-42 pushed a commit to opus-42/incubator-superset that referenced this pull request Nov 14, 2021
…17039)

* feat(config): Add none force-refresh option for auto refresh

* use general config

* fix lint issues

* last lint fix
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/S 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants