Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Copy with Remote Object Stores in datafusion-cli #9064

Merged
merged 17 commits into from
Feb 5, 2024

Conversation

manoj-inukolunu
Copy link
Contributor

@manoj-inukolunu manoj-inukolunu commented Jan 30, 2024

Which issue does this PR close?

Closes #8907 .

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Yes , added tests.

Are there any user-facing changes?

@manoj-inukolunu
Copy link
Contributor Author

Hello @alamb , Here is a draft for #8907 . I am yet to add tests. Currently DefaultObjectStoreRegistry

pub fn new() -> Self {
        let object_stores: DashMap<String, Arc<dyn ObjectStore>> = DashMap::new();
        object_stores.insert("file://".to_string(), Arc::new(LocalFileSystem::new()));
        Self { object_stores }
    }

registers only LocalFileSystem .
I am wondering if it should register other stores as well , like S3, GCS etc.

@manoj-inukolunu manoj-inukolunu marked this pull request as ready for review February 4, 2024 09:23
@@ -260,6 +256,36 @@ async fn exec_and_print(
Ok(())
}

async fn create_plan(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me. Thank you @manoj-inukolunu

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @manoj-inukolunu -- this makes sense to me and is a nice improvement

@alamb
Copy link
Contributor

alamb commented Feb 5, 2024

BTW I filed #9133 for a small follow on too

@alamb alamb merged commit c43c5f1 into apache:main Feb 5, 2024
22 checks passed
@alamb
Copy link
Contributor

alamb commented Feb 5, 2024

Thanks again @manoj-inukolunu

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Copy with Remote Object Stores in datafusion-cli
3 participants