Authentication: Enable user impersonation for Superset to HiveServer2 using hive.server2.proxy.user (a.fernandez) #3652

afernandez · 2017-10-12T00:19:25Z

Superset today has a config for impersonation when creating/editing a datasource.
When used with Presto, it actually creates a connection on behalf of the logged on user.
For Hive, we instead want to connect as the superuser (superset service account) but use the hive.server2.proxy.user property in the URI to enable impersonation.

Unit Tests passed.

cc @timifasubaa @mistercrunch

afernandez · 2017-10-12T00:41:32Z

Need to fix unit tests

afernandez · 2017-10-12T00:41:58Z

superset/db_engine_specs.py

@@ -184,6 +186,28 @@ def select_star(cls, my_db, table_name, schema=None, limit=100,
            sql = sqlparse.format(sql, reindent=True)
        return sql

+    @classmethod
+    def modify_url_for_impersonation(cls, url, impersonate_user, username):


Base class has methods for how to modify a URI and URL object for impersonation

afernandez · 2017-10-12T00:42:26Z

superset/db_engine_specs.py

+            url.query["hive_server2_proxy_user"] = username
+
+    @classmethod
+    def get_uri_for_impersonation(cls, uri, impersonate_user, username):


Python Engine Spec overrides the methods for how to modify a URI and URL object for impersonation

afernandez · 2017-10-12T00:42:41Z

superset/models/core.py

        self.sqlalchemy_uri = str(conn)  # hides the password

+    def get_effective_user(self, url, user_name=None):


Moved this to its own method

afernandez · 2017-10-12T00:43:03Z

superset/models/core.py

+            elif hasattr(g, 'user') and g.user.username:
+                effective_username = g.user.username
+        return effective_username
+
    def get_sqla_engine(self, schema=None, nullpool=False, user_name=None):


Renamed uri to url since it was of type SQLAlchemy.URL

afernandez · 2017-10-12T00:43:25Z

superset/models/core.py

+        self.db_engine_spec.modify_url_for_impersonation(url, self.impersonate_user, effective_username)
+
+        masked_url = self.get_password_masked_url(url)
+        logging.info("Database.get_sqla_engine(). Masked URL: {0}".format(masked_url))


Important to mask the URL while logging since it may contain a password

afernandez · 2017-10-12T00:43:43Z

superset/sql_lab.py

@@ -168,6 +168,7 @@ def handle_error(msg):
    session.merge(query)
    session.commit()
    logging.info("Set query to 'running'")
+    conn = None


In some code paths, conn was not defined

afernandez · 2017-10-12T00:44:04Z

superset/views/core.py

+            uri = db_engine.get_uri_for_impersonation(uri, impersonate_user, username)
+            masked_url = database.get_password_masked_url_from_uri(uri)
+
+            logging.info("Superset.testconn(). Masked URL: {0}".format(masked_url))


Important to mask the URL while logging since it may contain a password

afernandez · 2017-10-12T00:44:21Z

tests/core_tests.py

@@ -282,7 +282,8 @@ def test_testconn(self):
        # validate that the endpoint works with the password-masked sqlalchemy uri
        data = json.dumps({
            'uri': database.safe_sqlalchemy_uri(),
-            'name': 'main'
+            'name': 'main',
+            'impersonate_user': False


Unit test is failing since g.user.username is missing. Will fix soon.

afernandez

Initial annotations

afernandez · 2017-10-12T23:40:00Z

Unit tests passed,

Ran 168 tests in 268.075s

OK
Submitting coverage to coveralls.io...
Coverage submitted!
Job ##8777.1
https://coveralls.io/jobs/30207773

coveralls · 2017-10-12T23:55:36Z

Coverage increased (+0.08%) to 70.191% when pulling 4e9263e804d4297a94edf81aaeaf1b1f39cfd04d on afernandez:afernandez_impersonate into 52a9f27 on apache:master.

mistercrunch

Highlighted a few minor things but LGTM otherwise. Please lint according to our .pylinrc. We used to have automation with Landscape.io but switched it off since we move the repo to Apache...

mistercrunch · 2017-10-13T04:31:21Z

superset/db_engine_specs.py

+        :param impersonate_user: Bool indicating if impersonation is enabled
+        :param username: Effective username
+        """
+        if impersonate_user and username is not None:


NIT: I'd go without the is not None as None evals to False

mistercrunch · 2017-10-13T04:33:56Z

superset/db_engine_specs.py

+        backend_name = url.get_backend_name()
+
+        # Must be Hive connection, enable impersonation, and set param auth=LDAP|KERBEROS
+        if backend_name == "hive" and "auth" in url.query.keys() and impersonate_user is True and username is not None:


Some very long lines here. PEP8 says 80, our pylint say 90. If you want to lint your PR only you can git diff master... | flake8 --diff thought that's flake8 not pylint. There's also git-lint which can lint your diff as well.

mistercrunch · 2017-10-13T04:37:57Z

superset/db_engine_specs.py

+        :param impersonate_user: Bool indicating if impersonation is enabled
+        :param username: Effective username
+        """
+        if impersonate_user is True and "auth" in url.query.keys() and username is not None:


NIT: -is not None

…veServer2 using hive.server2.proxy.user (a.fernandez)

coveralls · 2017-10-13T23:18:00Z

Coverage decreased (-0.02%) to 70.364% when pulling f310817 on afernandez:afernandez_impersonate into bad6938 on apache:master.

afernandez · 2017-10-16T22:16:37Z

@mistercrunch would you be able to re-review these changes? Does the decrease in coveralls mean I need to add unit tests?

afernandez · 2017-10-20T01:51:17Z

Thanks for reviewing. I figured out how to do it without needing changes to PyHive. Will submit another PR soon.
@mistercrunch

…veServer2 using hive.server2.proxy.user (a.fernandez) (apache#3652)

afernandez changed the title ~~DI-1113. Authentication: Enable user impersonation for Superset to HiveServer2 using hive.server2.proxy.user (a.fernandez)~~ Authentication: Enable user impersonation for Superset to HiveServer2 using hive.server2.proxy.user (a.fernandez) Oct 12, 2017

afernandez commented Oct 12, 2017

View reviewed changes

afernandez force-pushed the afernandez_impersonate branch from eee12f1 to 4e9263e Compare October 12, 2017 23:39

mistercrunch requested changes Oct 13, 2017

View reviewed changes

DI-1113. Authentication: Enable user impersonation for Superset to Hi…

f310817

…veServer2 using hive.server2.proxy.user (a.fernandez)

afernandez force-pushed the afernandez_impersonate branch from 4e9263e to f310817 Compare October 13, 2017 23:03

mistercrunch approved these changes Oct 17, 2017

View reviewed changes

mistercrunch merged commit adef519 into apache:master Oct 17, 2017

michellethomas pushed a commit to michellethomas/panoramix that referenced this pull request May 24, 2018

DI-1113. Authentication: Enable user impersonation for Superset to Hi…

dd1454d

…veServer2 using hive.server2.proxy.user (a.fernandez) (apache#3652)

snyk-bot mentioned this pull request Aug 19, 2021

[Snyk] Security upgrade webpack-dev-server from 3.11.0 to 4.0.0 XirdigH/superset#25

Closed

m-ajay mentioned this pull request Sep 27, 2022

[Snyk] Fix for 9 vulnerabilities m-ajay/superset#30

Open

This was referenced Oct 6, 2022

[Snyk] Fix for 9 vulnerabilities m-ajay/superset#53

Open

[Snyk] Fix for 9 vulnerabilities m-ajay/superset#62

Open

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.20.5 labels Feb 27, 2024

m-ajay mentioned this pull request Mar 15, 2024

[Snyk] Fix for 2 vulnerabilities m-ajay/superset#271

Open

m-ajay mentioned this pull request Mar 22, 2024

[Snyk] Fix for 2 vulnerabilities m-ajay/superset#278

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Authentication: Enable user impersonation for Superset to HiveServer2 using hive.server2.proxy.user (a.fernandez) #3652

Authentication: Enable user impersonation for Superset to HiveServer2 using hive.server2.proxy.user (a.fernandez) #3652

afernandez commented Oct 12, 2017 •

edited

Loading

afernandez commented Oct 12, 2017

afernandez Oct 12, 2017

afernandez Oct 12, 2017

afernandez Oct 12, 2017

afernandez Oct 12, 2017

afernandez Oct 12, 2017

afernandez Oct 12, 2017

afernandez Oct 12, 2017

afernandez Oct 12, 2017 •

edited

Loading

afernandez left a comment

afernandez commented Oct 12, 2017

coveralls commented Oct 12, 2017 •

edited

Loading

mistercrunch left a comment

mistercrunch Oct 13, 2017

afernandez Oct 13, 2017

mistercrunch Oct 13, 2017

afernandez Oct 13, 2017

mistercrunch Oct 13, 2017

afernandez Oct 13, 2017

coveralls commented Oct 13, 2017 •

edited

Loading

afernandez commented Oct 16, 2017

afernandez commented Oct 20, 2017

		self.sqlalchemy_uri = str(conn) # hides the password

		def get_effective_user(self, url, user_name=None):

Authentication: Enable user impersonation for Superset to HiveServer2 using hive.server2.proxy.user (a.fernandez) #3652

Authentication: Enable user impersonation for Superset to HiveServer2 using hive.server2.proxy.user (a.fernandez) #3652

Conversation

afernandez commented Oct 12, 2017 • edited Loading

afernandez commented Oct 12, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

afernandez Oct 12, 2017 • edited Loading

Choose a reason for hiding this comment

afernandez left a comment

Choose a reason for hiding this comment

afernandez commented Oct 12, 2017

coveralls commented Oct 12, 2017 • edited Loading

mistercrunch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Oct 13, 2017 • edited Loading

afernandez commented Oct 16, 2017

afernandez commented Oct 20, 2017

afernandez commented Oct 12, 2017 •

edited

Loading

afernandez Oct 12, 2017 •

edited

Loading

coveralls commented Oct 12, 2017 •

edited

Loading

coveralls commented Oct 13, 2017 •

edited

Loading