Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: switch from conda to standard python and pip #1293

Merged
merged 28 commits into from
Nov 12, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
17f3fd6
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 9, 2021
c1cd7b1
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 9, 2021
16967cb
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 9, 2021
5ad9361
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 9, 2021
ba46d23
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 9, 2021
71f51e3
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 9, 2021
8aa0d6c
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
154a945
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
3cc72ae
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
7fced55
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
ab2367a
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
6d42973
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
9313cca
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
9c419c1
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
5ecf277
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
eb498e3
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
d48c2d5
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
c5ca45c
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
d38ca3a
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
9aca412
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
89ec2ef
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
2223902
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 10, 2021
560057e
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 11, 2021
89daf25
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 11, 2021
1ae3164
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 11, 2021
b8dff80
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 11, 2021
a94b48a
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 12, 2021
f3924d5
ci: switch from conda to standard python and pip
jdhughes-usgs Nov 12, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/ci-mf6.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ on:
- master
- develop
- 'release*'
- 'ci-pytest'
pull_request:
branches: [master, develop]

Expand Down
84 changes: 56 additions & 28 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ on:
- master
- develop
- 'release*'
- 'ci-pytest'
pull_request:
branches: [master, develop]

Expand Down Expand Up @@ -56,7 +55,6 @@ jobs:
run:
shell: bash

if: github.event_name != 'schedule'
steps:
# check out repo
- name: Checkout flopy repo
Expand Down Expand Up @@ -101,7 +99,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
os: [ubuntu-latest, macos-latest]
python-version: [3.9, 3.8, 3.7]
run-type: [std]
test-pth: [""]
Expand All @@ -114,9 +112,17 @@ jobs:
python-version: 3.9
run-type: script
test-path: run_scripts.py
- os: windows-latest
python-version: 3.9
run-type: std
test-path: ""
- os: windows-latest
python-version: 3.8
run-type: std
test-path: ""
defaults:
run:
shell: bash -l {0}
shell: bash

steps:
# check out repo
Expand All @@ -126,35 +132,51 @@ jobs:
- name: Get branch name
uses: nelonoel/branch-name@v1.0.1

- name: Cache Miniconda
uses: actions/cache@v2.1.0
env:
# Increase this value to reset cache if environment.yml has changed
CACHE_NUMBER: 2
with:
path: ~/conda_pkgs_dir
key: ${{ runner.os }}-${{ matrix.python-version }}-${{ matrix.run-type }}-${{ env.CACHE_NUMBER }}-${{ hashFiles('etc/environment.yml') }}

# Standard python fails on windows without GDAL installation
# Using custom bash shell ("shell: bash -l {0}") with Miniconda
- name: Setup Miniconda
uses: conda-incubator/setup-miniconda@v2.1.1
- name: Setup Python
uses: actions/setup-python@v2.2.2
with:
python-version: ${{ matrix.python-version }}
mamba-version: "*"
channels: conda-forge
auto-update-conda: true
activate-environment: flopy
use-only-tar-bz2: true

- name: Add jupyter and jupytext to notebooks run
if: matrix.run-type == 'nb'
- name: Upgrade pip and install wheel
run: |
python -m pip install --upgrade pip
pip install wheel

- name: Install Python packages on Windows
if: runner.os == 'Windows'
run: |
mamba install --name flopy jupyter jupytext
pip install -r etc/requirements.windows.pip.txt

- name: Add packages to flopy environment using mamba or conda
- name: Install Python packages on Linux and MacOS
if: runner.os != 'Windows'
run: |
mamba env update --name flopy --file etc/environment.yml
pip install -r etc/requirements.full.pip.txt

- name: Download Python packages with GDAL dependencies on Windows
if: runner.os == 'Windows'
shell: python
run: |
import os
from gohlkegrabber import GohlkeGrabber
gg = GohlkeGrabber()
wheel_path = os.path.join(".", "wheels")
os.makedirs(wheel_path, exist_ok=True)
gg.retrieve(wheel_path, "gdal")
gg.retrieve(wheel_path, "rasterio")
gg.retrieve(wheel_path, "fiona")

- name: Install Python packages with GDAL dependencies on Windows
if: runner.os == 'Windows'
run: |
ls -ltr ./wheels/
pip install ./wheels/GDAL*
pip install ./wheels/rasterio*
pip install ./wheels/Fiona*

- name: Add jupyter and jupytext to notebooks run
if: matrix.run-type == 'nb'
run: |
pip install jupyter jupytext

- name: Install pymake, xmipy, and flopy
run: |
Expand All @@ -167,10 +189,16 @@ jobs:
run: |
pytest -v ci_prepare.py

- name: Add executables directory to path
- name: Add executables directory to path on Linux and MacOS
if: runner.os != 'Windows'
run: |
echo "$HOME/.local/bin" >> $GITHUB_PATH

- name: Add executables directory to path on Windows
if: runner.os == 'Windows'
run: |
echo "C:\Users\runneradmin\.local\bin" >> $GITHUB_PATH

- name: Run pytest on autotest scripts
if: matrix.run-type == 'std'
working-directory: ./autotest
Expand Down
22 changes: 10 additions & 12 deletions .github/workflows/rtd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,24 +32,22 @@ jobs:
echo $GITHUB_REF
echo $GITHUB_EVENT_NAME

# Standard python fails on windows without GDAL installation
# Using custom bash shell ("shell: bash -l {0}") with Miniconda
- name: Setup Miniconda
uses: conda-incubator/setup-miniconda@v2.1.1
- name: Setup Python
uses: actions/setup-python@v2.2.2
with:
python-version: 3.9
mamba-version: "*"
channels: conda-forge
auto-update-conda: true
activate-environment: flopy

- name: Add jupyter and jupytext to scripts run
- name: Upgrade pip
run: |
python -m pip install --upgrade pip

- name: Install Python packages
run: |
mamba install --name flopy jupyter jupytext
pip install -r etc/requirements.full.pip.txt

- name: Add packages to flopy environment
- name: Add jupyter and jupytext to scripts run
run: |
mamba env update --name flopy --file etc/environment.yml
pip install jupyter jupytext

- name: Install pymake, xmipy, and flopy
run: |
Expand Down
34 changes: 34 additions & 0 deletions etc/requirements.full.pip.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
matplotlib>=1.4.0
numpy>=1.15.0

# testing
pylint
flake8
black
coverage
pytest
pytest-xdist
pytest-cov

# optional
appdirs
python-dateutil>=2.4.0
affine
scipy
pandas
netcdf4
pyshp>=2.0.0
rasterio
fiona
descartes
pyproj
shapely
geos
geojson
vtk

# external dependencies
requests

# MODFLOW API dependencies
bmipy
35 changes: 35 additions & 0 deletions etc/requirements.windows.pip.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
matplotlib>=1.4.0
numpy>=1.15.0

# testing
pylint
flake8
black
coverage
pytest
pytest-xdist
pytest-cov

# optional
appdirs
python-dateutil>=2.4.0
affine
scipy
pandas
netcdf4
pyshp>=2.0.0
descartes
pyproj
shapely
geos
geojson
vtk

# external dependencies
requests

# MODFLOW API dependencies
bmipy

# golke installer
gohlkegrabber