Skip to content

Commit

Permalink
[patch] Debug py version (#124)
Browse files Browse the repository at this point in the history
* Rewrite cached-miniforge to avoid `mamba update`

There is an awkward edge case when the update env explicitly specifies a range of python versions that this can override the python version specified as an argument to `conda-incubator/setup-miniconda`; although the update approach is suggested in that action's docs for caching an env, the approach here is safer. A new input variable is introduced to store the location of the env, but the old `miniforge-activate-environment` can still be specified for backwards compatibility (although what happens under the hood has changed).

* Add a bit of verbosity to the `write-environment` action
  • Loading branch information
liamhuber committed Jul 23, 2024
1 parent abb6ca6 commit 606379c
Show file tree
Hide file tree
Showing 2 changed files with 75 additions and 34 deletions.
106 changes: 73 additions & 33 deletions cached-miniforge/action.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'Setup cached miniforge'
description: 'Use conda environment files to build a cached (optional) python environment with miniforge, and local code with pip install'
description: 'Use conda environment files to build a cached (optional) python environment with miniforge, and (optionally) local code with pip install'

inputs:
python-version:
Expand All @@ -17,6 +17,10 @@ inputs:
description: 'A stringified integer for the cache number (can be used to force-bump the cache)'
default: '0'
required: false
env-path:
description: 'A full path to the environment, used for miniconda activate-environment and cache path'
default: 'cached-miniforge/my-env'
required: false
local-code-directory:
description: 'The location containing the code under development; targeted by pip install. If an empty string, local code will not be pip-installed at all!'
default: '.'
Expand All @@ -37,14 +41,14 @@ inputs:
description: 'conda-incubator/setup-miniconda argument'
default: strict
required: false
miniforge-activate-environment:
description: 'conda-incubator/setup-miniconda argument'
default: my-env
required: false
miniforge-use-mamba:
description: 'conda-incubator/setup-miniconda argument'
default: 'true'
required: false
miniforge-activate-environment:
description: 'DEPRECATED. Use env-path and include a path instead. If provided, will be used as the new env-path with cached-miniforge/ prepended as the path.'
default: ''
required: false
pip-install-versioneer:
description: 'Pip-install versioneer[toml]'
default: 'true'
Expand All @@ -65,40 +69,78 @@ runs:
- uses: pyiron/actions/write-environment@main
with:
env-files: ${{ inputs.env-files }}
- name: Setup Mambaforge
- name: Env name backwards compatibility patch
# Can be replaced with direct usage of `inputs.env-path` after a major version bump
id: env-path-backwards-compatibility
shell: bash -l {0}
run: |
if [ -z "${{ inputs.miniforge-activate-environment }}" ]; then
env_path=${{ inputs.env-path }}
else
env_path=cached-miniforge/${{ inputs.miniforge-activate-environment }}
fi
echo env-path=${env_path} >> $GITHUB_OUTPUT
- name: Calculate cache label info
if: inputs.use-cache == 'true'
id: cache-info
shell: bash -l {0}
run: |
pyversion_string=${{ inputs.python-version }}
pyversion_string=${pyversion_string/\./-}
env_string=${{ steps.env-path-backwards-compatibility.outputs.env-path }}
env_string=${env_string//\//-}
LABEL=${{ runner.os }}-${{ runner.arch }}-py-${pyversion_string}-${env_string}
HASH=${{ hashFiles(env.WRITE_ENVIRONMENT_OUTPUT_ENV_FILE) }}
TODAY=$(date +'%Y%m%d')
echo CASH_KEY=${LABEL}-conda-${HASH}-${TODAY}-${{ inputs.cache-number }} >> $GITHUB_OUTPUT
- name: Look for cached environment
if: inputs.use-cache == 'true'
uses: actions/cache/restore@v4
id: look-for-cache
with:
path: ${{ steps.env-path-backwards-compatibility.outputs.env-path }}
key: ${{ steps.cache-info.outputs.CASH_KEY }}
lookup-only: true
- name: Install using cached env
if: inputs.use-cache == 'true' && steps.look-for-cache.outputs.cache-hit == 'true'
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: ${{ inputs.miniforge-variant }}
miniforge-version: ${{ inputs.miniforge-version }}
channels: ${{ inputs.miniforge-channels }}
channel-priority: ${{ inputs.miniforge-channel-priority }}
activate-environment: ${{ steps.env-path-backwards-compatibility.outputs.env-path }}
use-mamba: ${{ inputs.miniforge-use-mamba }}
- name: Load cached environment
if: inputs.use-cache == 'true' && steps.look-for-cache.outputs.cache-hit == 'true'
uses: actions/cache/restore@v4
with:
path: ${{ steps.env-path-backwards-compatibility.outputs.env-path }}
key: ${{ steps.cache-info.outputs.CASH_KEY }}
- name: Build environment from file
if: inputs.use-cache != 'true' || steps.look-for-cache.outputs.cache-hit != 'true'
uses: conda-incubator/setup-miniconda@v3
with:
python-version: ${{ inputs.python-version }}
miniforge-variant: ${{ inputs.miniforge-variant }}
miniforge-version: ${{ inputs.miniforge-version }}
channels: ${{ inputs.miniforge-channels }}
channel-priority: ${{ inputs.miniforge-channel-priority }}
activate-environment: ${{ inputs.miniforge-activate-environment }}
activate-environment: ${{ steps.env-path-backwards-compatibility.outputs.env-path }}
use-mamba: ${{ inputs.miniforge-use-mamba }}
- name: Calculate cache label info
id: cache-info
shell: bash -l {0}
run: |
pyversion_string=${{ inputs.python-version }}
pyversion_string=${pyversion_string/\./-}
echo LABEL=${{ runner.os }}-${{ runner.arch }}-py-${pyversion_string} >> $GITHUB_OUTPUT
echo HASH=${{ hashFiles(env.WRITE_ENVIRONMENT_OUTPUT_ENV_FILE) }} >> $GITHUB_OUTPUT
echo "TODAY=$(date +'%Y%m%d')" >> $GITHUB_OUTPUT
- uses: actions/cache@v4
id: cache
if: ${{ inputs.use-cache == 'true' }}
env:
CACHE_NUMBER: ${{ inputs.cache-number }}
environment-file: ${{ env.WRITE_ENVIRONMENT_OUTPUT_ENV_FILE }}
- name: Cache new env
if: inputs.use-cache == 'true' && steps.look-for-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v4
id: cache-env
with:
path: ${{ env.CONDA }}/envs
key: ${{ steps.cache-info.outputs.LABEL }}-conda-${{ steps.cache-info.outputs.HASH }}-${{ steps.cache-info.outputs.TODAY }}-${{ env.CACHE_NUMBER }}
- name: Update environment
if: inputs.use-cache != 'true' || steps.cache.outputs.cache-hit != 'true'
path: ${{ steps.env-path-backwards-compatibility.outputs.env-path }}
key: ${{ steps.cache-info.outputs.CASH_KEY }}
- name: Display env info
shell: bash -l {0}
run: mamba env update -n ${{ inputs.miniforge-activate-environment }} -f ${{ env.WRITE_ENVIRONMENT_OUTPUT_ENV_FILE }}
- name: Conda list
shell: bash -l {0}
run: conda list
run: |
conda info
conda list
- name: Install versioneer
if: inputs.local-code-directory != '' && inputs.pip-install-versioneer == 'true'
shell: bash -l {0}
Expand All @@ -107,10 +149,8 @@ runs:
- name: Install local code without build isolation
if: inputs.local-code-directory != '' && inputs.no-build-isolation == 'true'
shell: bash -l {0}
run: |
pip install --no-deps ${{ inputs.local-code-directory }} --no-build-isolation
run: pip install --no-deps ${{ inputs.local-code-directory }} --no-build-isolation
- name: Install local code with build isolation
if: inputs.local-code-directory != '' && inputs.no-build-isolation != 'true'
shell: bash -l {0}
run: |
pip install --no-deps ${{ inputs.local-code-directory }}
run: pip install --no-deps ${{ inputs.local-code-directory }}
3 changes: 2 additions & 1 deletion write-environment/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,5 @@ runs:
shell: bash -l {0}
run: |
python $GITHUB_ACTION_PATH/../.support/condamerge.py ${{ inputs.env-files }} > ${{ inputs.output-env-file }}
echo WRITE_ENVIRONMENT_OUTPUT_ENV_FILE=${{ inputs.output-env-file }} >> $GITHUB_ENV
echo WRITE_ENVIRONMENT_OUTPUT_ENV_FILE=${{ inputs.output-env-file }} >> $GITHUB_ENV
cat ${{ inputs.output-env-file }}

0 comments on commit 606379c

Please sign in to comment.