You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When writing to an HDFS sink (via saveAsHdfsAvro() for example) with username parameter, some of the workers fail to write because they still operate as "root" user and get a permission error. Either the workers do not all receive the username, or they are late to change the environment variable (maybe a serialization problem).
The text was updated successfully, but these errors were encountered:
When writing to an HDFS sink (via
saveAsHdfsAvro()
for example) with username parameter, some of the workers fail to write because they still operate as "root" user and get a permission error. Either the workers do not all receive the username, or they are late to change the environment variable (maybe a serialization problem).The text was updated successfully, but these errors were encountered: