Use refresh instead of force refresh during auto-refresh

[yongchand]

Is your feature request related to a problem? Please describe.
I know that currently an auto-refresh of dashboard is force refresh which is removing cache and querying again. I personally think this is very inefficient and it is better to use regular refresh. Let us say user A entered in dashboard with 1 min auto refresh at 12:00:00, and user B entered at 12:00:30. Then, force refresh will occur at 12:01:00, and also 12:01:30 - which seems unnecessary and not using cache at all. This is especially stressful when auto refresh interval is shorter and multiple users enter.

Describe the solution you'd like
I would propose Superset auto-refresh to use (normal) refresh instead of force refresh. We (our data team) customized Superset to use (normal) refresh for auto refresh instead of force refresh. This seemed way better since dashboard will call cache if it exists, and will query if it does not. (And of course, users can choose own cache timeout for chart, dataset, database) We have done stress test with 100 users (with one second term of entering dashboard, and 30 seconds of auto refresh), and found out that with the original setting (force refresh) Superset will create a problem within 5 minutes, while with our new setting (normal refresh) Superset will perform properly.

Describe alternatives you've considered
We can possibly make an option or config where users can choose whether to force-refresh or normal-refresh for auto refresh in dashboard.

Additional context
I am willing to create PR based on what we have customized already
Related slack context: https://apache-superset.slack.com/archives/CK1P3L23E/p1633108779190200

[yongchand]

Any thoughts on this issue :) ? @betodealmeida

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[rusackas]

I think this was effectively closed by #17039 but if not, I'm tempted to close it as stale anyway. If there are additional features that need discussion or development, I think we should move this to an Ideas thread on the Discussions page, and gather input/support there instead. Thank you!