-
Notifications
You must be signed in to change notification settings - Fork 129
Development Setup
Follow our github action workflow to spin up the development stack in a way that is continuously tested.
https://github.com/ansible/galaxy_ng/blob/master/.github/workflows/ci_standalone.yml
Our containerized development environment configuration is loaded from the .compose.env
script. You'll first need to configure it, and the easiest way to do that is by copying an example script .compose.env.example
:
$ cp .compose.env.example .compose.env
By default, the development environment is configured to run in insights mode, which requires a 3rd party authentication provider. If you're working outside of the Red Hat cloud platform, you'll want to switch it to standalone mode by modifying your .compose.env
file, and setting the COMPOSE_PROFILE
variable to standalone
, as shown in the following example:
COMPOSE_PROFILE=standalone
If you want to run in standalone mode while using Keycloak to provide single sign-on with a pre-populated LDAP server you'll want to switch it to standalone-keycloak mode by modifying your .compose.env
file, and setting the COMPOSE_PROFILE
variable to standalone-keycloak
, as shown in the following example:
COMPOSE_PROFILE=standalone-keycloak
If you would like to develop using the UI, simply do the following:
-
Set
ANSIBLE_HUB_UI_PATH
in your.compose.env
file to point to the location of the cloned UI repo. Absolute paths aren't required, but they're easier to set up. If you want to use a relative path, it has to be relative todev/docker-compose.yml
ANSIBLE_HUB_UI_PATH='/absolute/path/to/ansible-hub-ui'
-
Complete the rest of the steps in the next section. Once everything is running the UI can be accessed at http://localhost:8002
NOTE: There is no default user or password. make docker/loaddata will create an admin/admin user.
Skip this step if you don't have access to the Red Hat VPN.
If you want to be able to run the app in insights mode you need to add the following in your /etc/hosts
file.
127.0.0.1 prod.foo.redhat.com
127.0.0.1 stage.foo.redhat.com
127.0.0.1 qa.foo.redhat.com
127.0.0.1 ci.foo.redhat.com
To access the UI when running in insights mode:
- Connect to the Red Hat VPN
- Navigate to https://ci.foo.redhat.com:1337/beta/ansible/automation-hub
- You'll need a Red Hat username and password to authenticate with the dev Red Hat SSO server. Anyone on the Galaxy team should be able to provide you with one.
Next, run the following steps to build your local development environment:
-
Build docker image:
$ ./compose build
-
Start database and redis services:
$ ./compose up -d postgres redis
-
Run migrations:
$ ./compose run --rm api manage migrate
-
Start services
In background:
$ ./compose up -d
In foreground:
$ ./compose up
If using standalone-keycloak
you will need to initialize your Keycloak instance before running migrations and starting the remaining services.
-
Start the Keycloak instance and dependencies
$ ./compose up -d keycloak kc-postgres ldap
-
Bootstrap the Keycloak instance with a Realm and Client then capture the needed public key
$ ansible-playbook ./dev/standalone-keycloak/keycloak-playbook.yaml # Try again if it fails at the first run, services might not be available yet.
-
Update your
.compose.env
file with the public key found at the end of the playbook runPULP_SOCIAL_AUTH_KEYCLOAK_PUBLIC_KEY="keycloak-public-key"
After the standard development set up steps, when you access http://localhost:8002 it will redirect to a Keycloak Open ID Connect flow login page, where you can login with one of the development SSO user's credentials (the password is the username). If you want to login with non-Keycloak users, you need to use the Django admin console.
If you want to login as a superuser, you can do one of two things:
- Login to the Django admin console with the admin user
- Login to the Keycloak instance with admin/admin to edit the LDAP user's roles: Choose a development SSO user, select Role Mappings > Client Roles > automation-hub and add the
hubadmin
role. A user is associated with the appropriate group(s) using the user_group pipeline.
To push images into the container image registry hosted by galaxy_ng (via pulp_container), you need to tag an image first to tell Docker or Podman that you want to associate the image with the registry. On a local development setup, the pulp_container runs along with the Galaxy API on port 5001.
Tag am image like this:
docker image tag <IMAGE_ID> localhost:5001/<NAME>:<VERSION>
or, to associate with a namespace:
docker image tag <IMAGE_ID> localhost:5001/<NAMESPACE>/<NAME>:<VERSION>
And then push the image and the engine will upload it to the now-associated registry:
docker push localhost:5001/testflat
When running docker environment, the project's parent directory is mounted into container as /app
. All projects listed in DEV_SOURCE_PATH
environment variable are installed in editable mode (pip install -e
) in the container. To load additional projects such as galaxy-importer
or pulp_ansible
into the container from host file system you should clone them into the parent directory relative to your galaxy_ng
working copy location.
For example you want to work on galaxy-importer
project and run development environment with your changes made locally. Assuming your current working directory is galaxy_ng
.
-
Clone
galaxy-importer
to parent directory:$ git clone https://github.com/ansible/galaxy-importer.git ../galaxy-importer
-
Add
galaxy-importer
toDEV_SOURCE_PATH
variable in your.compose.env
file:export DEV_SOURCE_PATH='galaxy_ng:galaxy-importer'
-
Recreate your development environment:
$ ./compose down $ ./compose up -d
-
Clone locally galaxy_ng, pulpcore and pulp_ansible all the repos must be located at the same directory level.
cd ~/projects/ git clone https://github.com/pulp/pulpcore git clone https://github.com/pulp/pulp_ansible git clone https://github.com/ansible/galaxy_ng # and optionally git clone https://github.com/ansible/ansible-hub-ui git clone https://github.com/ansible/galaxy_importer
-
- Checkout to desired branches.
-
pulp_ansible master is compatible with a specific range of pulpcore
versions. So the recommended is to checkout to specific branch or tag following the contraints defined on pulp_ansible/requirements.txt or leave it checked out to master if you know it is compatible with the pulp_ansible branch you have. Example:
This is also possible to checkout to specific pull-requests by its refs/pull/id.cd ~/projects/pulpcore git checkout 3.9.0
-
Edit the galaxy_ng/.compose.env file.
DEV_SOURCE_PATH refers to the repositories you cloned locally, the order is important from the highest to the low dependecy, otherwise pip will raise version conflicts.cd ~/projects/galaxy_ng cat .compose.env COMPOSE_PROFILE=standalone DEV_SOURCE_PATH='pulpcore:pulp_ansible:galaxy_ng' LOCK_REQUIREMENTS=0
So pulpcore is a dependency to pulp_ansible which is a dependency to galaxy_ng, this order must be respected on DEV_SOURCE_PATH variable.
LOCK_REQUIREMENTS when set to 0 it tells docker to bypass the install of pinned requirements and rely only on packages defined on setup.py for each repo.
-
Run ./compose build to make those changes effective.
-
Run desired compose command: ./compose up, ./compose run etc...
The canonical source of truth for the 'version' is now in setup.cfg in the bumpversion stanza:
[bumpversion]
current_version = 4.3.0.dev
To update version, it is recommended to "bump" the version instead of explicitly specifying it.
Use bump2version to increment the 'version' string wherever it is needed.
It can 'bump' the 'patch', 'minor', 'major' version components.
There are also Makefile targets for bumping versions. To do a 'patch' version bump, for example:
$ make dev/bumpversion-patch
The above command will rev the 'patch' level and update all the files that use it.
Note: Currently, the bump2version config does not git commit or git tag the changes. So after bumping the version, you need to commit the changes and tag manually.
$ git commit -v -a
$ git tag $NEWVERSION
bump2version can also do this automatically if we want to enable it.
https://github.com/ansible/galaxy_ng/wiki/Debugging-with-PDB
To set your own galaxy-importer.cfg, add something like this to /dev/Dockerfile.base
:
RUN mkdir /etc/galaxy-importer \
&& printf "[galaxy-importer]\n \
REQUIRE_V1_OR_GREATER = True\n \
LOG_LEVEL_MAIN = INFO\n" | tee /etc/galaxy-importer/galaxy-importer.cfg
Sponsored by Red Hat, Inc.