Load CSV from backend storage (GCS)

[YannBrrd]

[SIP] Proposal to add a 'Load CSV from...' feature

Motivation

Providing a way to load CSV from different storages (first in my mind is GCS).

Proposed Change

On top of providing a way to load CSV from user interface, allow to load a file from a backend storage such as GCS or other backends from cloud providers

New or Changed Public Interfaces

Adding a 'Load CSV from...' button

New dependencies

I'd tend to use libcloud, from Apache.

Migration Plan and Compatibility

None

Rejected Alternatives

None, AFAIK.

[issue-label-bot]

Issue-Label Bot is automatically applying the label #enhancement to this issue, with a confidence of 0.95. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[willbarrett]

Thank you for opening the SIP - I'd love to understand the usecase better here. What is the motivation for adding the ability to load CSVs from cloud locations? What user behavior are you trying to support. Thanks!

[mistercrunch]

If it fits in a few lines it probably does not require the formal SIP process.
#5602

I think we should completely rebuild the CSV uploader in React.

[YannBrrd]

Use case is basic, in a corporante environnement, team would share a CSV on a backend storage and then could be analyzed using SuperSet without getting the data out of cloud... Does it make sense ?

[willbarrett]

Yes, that makes sense @YannBrrd. Go for it. Be aware that this system is going to be rebuilt from scratch at some point in the near future, so please attempt to make your addition modular and loosely coupled. That will help us avoid redoing your work when we migrate the feature.

[YannBrrd]

I've been looking at how you connect to BigQuery. It is an application level access if I got it well. Might not it be better/an option to have OAuth2 ? Doing this, I'd inherit accesses for current user...

[willbarrett]

Adding an option for OAuth2 connections to BigQuery would be a great addition. It might be worth a SIP to investigate the approach before moving forward. I do know some organizations have been able to use a service account and impersonation to achieve the same access controls.

[YannBrrd]

Once done for BQ it will be basic for GCS. Have access rto the method others used?

[willbarrett]

@YannBrrd unfortunately that code is closed-source, so I cannot share it.

[YannBrrd]

Fair enough.

I'm drowning in Google Python APIs. 😃

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.