Move API OIDC/OAuth2 support from API to ManageIQ core

[jvlcek]

Fixes #19865

This PR moves the OpenID-Connect support for the API, as implemented with PR 737 and PR 747, from being implemented directly in the ManageIQ API code base to the ManageIQ core authenticator code base.

This PR must be merged along with the associated ManageIQ/manageiq-api
PR 772

Steps for Testing/QA

1. Configure an appliance for OpenID-connect authentication [1]
2. Log in to the appliance using the UI
3. Request the users Access Token
4. Accessing ManageIQ API with the ACCESS_TOKEN
5. Accessing ManageIQ API with USER and PASSWORD

The below shell commands outline how to perform steps 3 through 5:

# Request the users Access Token
TOKEN_ENDPOINT="http://${KEYCLOAK_SERVER}:8080/auth/realms/${KEYCLOAK_REALM}/protocol/openid-connect/token"
KEYCLOAK_CLIENT_ID=<the value of OIDCClientSecret from  /etc/httpd/conf.d/manageiq-external-auth-openidc.conf>
OID_CLIENT_SECRET=<the value of OIDCCLientID from  /etc/httpd/conf.d/manageiq-external-auth-openidc.conf>
OID_CLIENT_HOST=<Full DNS hostname for ManageIQ appliance>

RES=`curl -L --user ${USER}:${PASSWORD} -X POST \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=password" \
  -d "client_id=${KEYCLOAK_CLIENT_ID}" \
  -d "client_secret=${OID_CLIENT_SECRET}" \
  -d "username=$USER" \
  -d "password=$PASSWORD" \
  $TOKEN_ENDPOINT`
ACCESS_TOKEN=`echo $RES | jq -r '.access_token'`

# Accessing ManageIQ API with the ACCESS_TOKEN
curl -L -vvv -k -X GET -H "Authorization: Bearer ${ACCESS_TOKEN}" https://${OID_CLIENT_HOST}/api/users  | jq

# Accessing ManageIQ API with USER and PASSWORD
curl -L -vvv -k --user ${USER}:${PASSWORD} -X GET -H "Accept: application/json" https://${OID_CLIENT_HOST}/api/users | jq

[1] https://www.manageiq.org/docs/reference/latest/auth/openid_connect

[jvlcek]

@miq-bot assign @abellotti
@miq-bot add_label core/authentication

[jvlcek]

@miq-bot add_label refactoring

[jvlcek]

Note: It might be possible to simplify this solution, and eliminate most if not all of the code introduced in this PR, by leveraging OpenID-Connect/OAuth2 setup in the mod_auth_openidc configuration files. Issue 19866

Spec test will be added f it turns out not to be possible to leveraging OpenID-Connect/OAuth2 setup in the mod_auth_openidc configuration files.

[abellotti]

don't we need to check that username is present instead ?

[jvlcek]

Will do! Thank you.

[jvlcek]

@bdunne, @Fryguy and @abellotti

I believe I've addressed all of your concerns. I would find it inconceivable for you to uncover any other issues with this PR.

Thank you for your help and feedback.

[jvlcek]

I apologies for the full file difference for httpd/oauth2.rb The namespace changes identified by @Fryguy caused the file indentation to change.

[bdunne]

Can you add some tests?

[jvlcek]

@bdunne Thank you for the review input.

As noted in comment 2 I plan to add tests if it turns out this code can not easily be replaced by OIDC configuration. I don't want to spend time adding tests now for code that just might be removed and the priority is to try to remove it.

Note: It might be possible to simplify this solution, and eliminate most if not all of the code introduced in this PR, by leveraging OpenID-Connect/OAuth2 setup in the mod_auth_openidc configuration files. Issue 19866

Spec test will be added f it turns out not to be possible to leveraging OpenID-Connect/OAuth2 setup in the mod_auth_openidc configuration files.

[kbrock]

This is a moved from api to core.
There are no tests over there.
The shorter term goal is to delete this.

I say merge as is.

You game with this @bdunne ?

[miq-bot]

Checked commits jvlcek/manageiq@e8f495c~...a517b55 with ruby 2.5.7, rubocop 0.69.0, haml-lint 0.20.0, and yamllint
2 files checked, 0 offenses detected
Everything looks fine. 🍪

[bdunne]

• This is no longer a simple move. It's been significantly changed in the process
• Why are we moving it if the plan is to replace it with something else in the short term?
• Would the high-level tests (pass these HTTP headers which validate a user is logged in) be significantly different with the replacement module?

[Fryguy]

This is no longer a simple move. It's been significantly changed in the process

I don't think it was characterized as a simple move. This was a cleanup effort for the original hacked in feature in order to put it in a proper location, and ideally into a dedicated concern should it be removed later. Not much of the actual code should have changed except in light of cleanup.

Why are we moving it if the plan is to replace it with something else in the short term?

We are not sure how much effort it will be to replace with the Apache based thing and would prefer to have this "clean" in the jansa release. If we can get the Apache based thing, then yeah, this will go away. It was a toss-up choice and we chose cleanup followed by Apache-based.

Would the high-level tests (pass these HTTP headers which validate a user is logged in) be significantly different with the replacement module?

I don't think so, and I agree with you that it's probably a good idea to have tests for this stuff... I'd be ok with waiting on merge for some high-level tests. Might have to use a bit of mocking because the way it works it to reach out to a remote identity system.

[Fryguy]

@jvlcek Can you please run cross-repo tests with your multiple PRs as well as UI?

[jvlcek]

@bdunne Thank you for your perspective! I'm just trying to avoid wasting time on specs now if I can figure out how to get rid of this code.

@Fryguy cross-repo tests here: ManageIQ/manageiq-cross_repo-tests#85

[jvlcek]

@bdunne @Fryguy @abellotti and @kbrock Thoughts on merging this? The cross-repo tests passed. Are folks OK with adding specs if the alternative approach turns out not to be possible?

[bdunne]

@jvlcek Does this work roughly the same way as the potential replacement (HTTPD passes us some headers and that means the user is authenticated)?

[bdunne]

It looks like there are tests around the other parts of the authenticate conditional. Added #19866 (comment) as a reminder to add more tests if the replacement is not viable.

[simaishi]

Jansa backport details:

$ git log -1
commit 447b26fa59e35147aea78b2a8ecbe348ba77e06a
Author: Brandon Dunne <bdunne@redhat.com>
Date:   Fri Mar 13 12:05:30 2020 -0400

    Merge pull request #19936 from jvlcek/oidc_api_to_core_issue_19865

    Move API OIDC/OAuth2 support from API to ManageIQ core

    (cherry picked from commit d8914fb455965c86b26ab2da9484ab2133874e3b)