Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor PullRequest Image Building and Testing GitHub Actions CI #118

Merged
merged 27 commits into from
Feb 6, 2020
Merged

Refactor PullRequest Image Building and Testing GitHub Actions CI #118

merged 27 commits into from
Feb 6, 2020

Conversation

scottyhq
Copy link
Member

@scottyhq scottyhq commented Feb 3, 2020

This PR also re-configures the Pull Request GitHub Action workflow to pass the base-notebook image between jobs as an "artifact". This is avoids the need to authenticate with docker hub, which is inconvenient for secrets management - PRs often come from forked repositories, but those workflows DO NOT have access to secrets stored in this repository (https://github.51.almunity/t5/GitHub-Actions/Make-secrets-available-to-builds-of-forks/td-p/30678/page/2 ).

This pull request supersedes:
#120
#121
#123

@scottyhq
Copy link
Member Author

scottyhq commented Feb 5, 2020

seeing a lot of UnsatisfiableError: The following specifications were found to be incompatible with the existing python installation in your environment: in the build logs for pangeo-notebook and pangeo-esip.

  - cmocean -> python[version='2.7.*|3.4.*|3.5.*|3.6.*']
  - eofs -> python[version='2.7.*|3.4.*|3.5.*|3.6.*']
  - palettable -> python[version='2.7.*|3.4.*|3.5.*|3.6.*']
  - watermark -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0']
  - xrft -> python[version='2.7.*|3.5.*|3.6.*']

Your python: python=3.7
  - eofs -> python[version='2.7.*|3.4.*|3.5.*|3.6.*']
  - geolinks -> python[version='2.7.*|3.4.*|3.5.*|3.6.*']
  - pocean-core -> python[version='2.7.*|3.5.*|3.6.*']
  - pyoos -> python[version='2.7.*|3.4.*|3.5.*|3.6.*']
  - utide -> python[version='2.7.*|3.4.*|3.5.*|3.6.*']
  - xrft -> python[version='2.7.*|3.5.*|3.6.*']

Eventually leading to failed conda installs with

Traceback (most recent call last):
  File "/usr/local/bin/r2d_overlay.py", line 167, in <module>
    main()
  File "/usr/local/bin/r2d_overlay.py", line 161, in main
    build()
  File "/usr/local/bin/r2d_overlay.py", line 122, in build
    ['/bin/bash', '-c', command], preexec_fn=applicator._pre_exec
  File "/srv/conda/envs/notebook/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/bash', '-c', 'conda env update -p /srv/conda/envs/notebook -f /home/jovyan/.onbuild-child/binder/environment.yml']' returned non-zero exit status 1.
The command '/bin/sh -c /usr/local/bin/r2d_overlay.py build' returned a non-zero code: 1

This was possibly due to a release of Repo2Docker today, so I'm pinning to the previous version 0.10 to reduce the number of changes https://github.com/jupyter/repo2docker/releases/tag/0.11.0

@scottyhq scottyhq changed the title Bump dask versions Refactor PR CI, Bump base-notebook versions Feb 5, 2020
@scottyhq scottyhq changed the title Refactor PR CI, Bump base-notebook versions Refactor PullRequest Image Building and Testing GitHub Actions CI Feb 5, 2020
@scottyhq
Copy link
Member Author

scottyhq commented Feb 5, 2020

To speed up building I've removed the ML-related images from pangeo-notebook. I'm thinking we keep those in a separate ML-focused notebook #119

@scottyhq
Copy link
Member Author

scottyhq commented Feb 5, 2020

Came across a failed build due to conda network error:

2020-02-05T20:34:59.5638308Z Step 2/2 : FROM pangeo/base-notebook-onbuild:$VERSION
2020-02-05T20:34:59.5643353Z # Executing 5 build triggers
2020-02-05T20:35:11.0651653Z  ---> Running in fb90ae00402e
2020-02-05T20:35:38.1972491Z Removing intermediate container fb90ae00402e
2020-02-05T20:35:38.3746330Z  ---> Running in d038294a9c4a
2020-02-05T20:36:40.4058508Z Collecting package metadata (repodata.json): ...working... failed
2020-02-05T20:36:40.4064572Z �[91m
2020-02-05T20:36:40.4065012Z CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://conda.anaconda.org/conda-forge/linux-64/repodata.json>
2020-02-05T20:36:40.4065302Z Elapsed: -
2020-02-05T20:36:40.4065361Z 
2020-02-05T20:36:40.4065475Z An HTTP error occurred when trying to retrieve this URL.
2020-02-05T20:36:40.4065598Z HTTP errors are often intermittent, and a simple retry will get you on your way.
2020-02-05T20:36:40.4066267Z ConnectionError(ReadTimeoutError("HTTPSConnectionPool(host='conda.anaconda.org', port=443): Read timed out."))
2020-02-05T20:36:40.4066347Z 
2020-02-05T20:36:40.4066415Z 
2020-02-05T20:36:40.7354874Z �[0m�[91mTraceback (most recent call last):
2020-02-05T20:36:40.7356440Z   File "/usr/local/bin/r2d_overlay.py", line 147, in <module>
2020-02-05T20:36:40.7356977Z     main()
2020-02-05T20:36:40.7357176Z   File "/usr/local/bin/r2d_overlay.py", line 141, in main
2020-02-05T20:36:40.7357308Z     build()
2020-02-05T20:36:40.7357456Z   File "/usr/local/bin/r2d_overlay.py", line 109, in build
2020-02-05T20:36:40.7357891Z     ['/bin/bash', '-c', command], preexec_fn=applicator._pre_exec
2020-02-05T20:36:40.7358052Z   File "/srv/conda/envs/notebook/lib/python3.7/subprocess.py", line 347, in check_call
2020-02-05T20:36:40.7358220Z     raise CalledProcessError(retcode, cmd)
2020-02-05T20:36:40.7358661Z subprocess.CalledProcessError: Command '['/bin/bash', '-c', 'conda env update -p /srv/conda/envs/notebook -f /home/jovyan/.onbuild-child/binder/environment.yml']' returned non-zero exit status 1.
2020-02-05T20:36:40.8872912Z The command '/bin/sh -c /usr/local/bin/r2d_overlay.py build' returned a non-zero code: 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant