Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDFS username not propagated to all workers when writing to Hadoop sink #124

Closed
andrewsmartin opened this issue Apr 28, 2016 · 0 comments
Closed
Assignees
Labels
bug Something isn't working
Milestone

Comments

@andrewsmartin
Copy link
Contributor

When writing to an HDFS sink (via saveAsHdfsAvro() for example) with username parameter, some of the workers fail to write because they still operate as "root" user and get a permission error. Either the workers do not all receive the username, or they are late to change the environment variable (maybe a serialization problem).

@andrewsmartin andrewsmartin self-assigned this Apr 28, 2016
@andrewsmartin andrewsmartin added the bug Something isn't working label Apr 28, 2016
@nevillelyh nevillelyh added this to the 0.2.0 milestone Apr 29, 2016
andrewsmartin added a commit that referenced this issue May 14, 2016
andrewsmartin added a commit that referenced this issue May 17, 2016
andrewsmartin added a commit that referenced this issue May 17, 2016
andrewsmartin added a commit that referenced this issue May 17, 2016
andrewsmartin added a commit that referenced this issue May 17, 2016
andrewsmartin added a commit that referenced this issue May 17, 2016
andrewsmartin added a commit that referenced this issue May 17, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants