Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Integration Testing] PEcAn Docker Stack Run : Single Site, Single Model #3345

Open
wants to merge 40 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
6934bc9
added model run
meetagrawal09 Jul 27, 2024
652d81e
trial
Aug 1, 2024
8b36096
trial
Aug 1, 2024
c70a06e
Accepted Suggestions
meetagrawal09 Aug 3, 2024
2e39f1c
Added wait time
meetagrawal09 Aug 3, 2024
9db5d2e
Bug fix
meetagrawal09 Aug 3, 2024
d194e21
Shifted to executor
meetagrawal09 Aug 3, 2024
b94bda7
Merge branch 'develop' into model-run-workflow
meetagrawal09 Aug 3, 2024
e8a359a
Merge branch 'develop' into model-run-workflow
meetagrawal09 Aug 8, 2024
266f0c6
Trying run by avoiding creating a user
meetagrawal09 Aug 8, 2024
d3557ef
Create docker compose override
meetagrawal09 Aug 15, 2024
c8ba82e
Remove docker images
meetagrawal09 Aug 15, 2024
a389dd6
Merge branch 'develop' into model-run-workflow
meetagrawal09 Aug 15, 2024
c78e999
Trying free-disk-space action
meetagrawal09 Aug 15, 2024
3c6d30a
typo
meetagrawal09 Aug 15, 2024
51060d7
trying rstudio container
meetagrawal09 Aug 16, 2024
d845de3
trial
meetagrawal09 Aug 16, 2024
3c51a46
trial
meetagrawal09 Aug 16, 2024
d3f835e
Removing entry for volume
meetagrawal09 Aug 22, 2024
363edd8
moved to executor
meetagrawal09 Aug 22, 2024
724160b
bug fix
meetagrawal09 Aug 22, 2024
76e5607
bug fix
meetagrawal09 Aug 22, 2024
d9ae803
Merge branch 'develop' into model-run-workflow
meetagrawal09 Aug 24, 2024
eb896d1
clean-up
meetagrawal09 Sep 5, 2024
0b6b71d
bug fix
meetagrawal09 Sep 5, 2024
87752b4
updated output dir
meetagrawal09 Sep 5, 2024
6ffefd2
Removed sleep
meetagrawal09 Sep 5, 2024
6771566
Keeping sleep with a comment
meetagrawal09 Sep 5, 2024
e4007f5
Update workflow name
meetagrawal09 Sep 5, 2024
998cce7
Merge branch 'develop' into model-run-workflow
meetagrawal09 Sep 5, 2024
b4f6d09
updated settings
meetagrawal09 Sep 5, 2024
824bb65
Merge branch 'develop' into model-run-workflow
meetagrawal09 Sep 8, 2024
a06e01b
Merge branch 'develop' into model-run-workflow
meetagrawal09 Sep 12, 2024
5896395
Merge branch 'develop' into model-run-workflow
infotroph Sep 12, 2024
c6d38df
Removing sleep -> relying on healthchecks
meetagrawal09 Sep 14, 2024
8556f5b
Adding sleep for testing
meetagrawal09 Sep 14, 2024
0bc2c78
Using model name
meetagrawal09 Sep 15, 2024
a55143d
Running add models script
meetagrawal09 Sep 16, 2024
69c27d6
Run add.models.sh
meetagrawal09 Sep 21, 2024
a949401
Merge branch 'develop' into model-run-workflow
meetagrawal09 Sep 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/workflows/model-run.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name : Sipnet Model Run
on :
pull_request:

# allow manual triggering
workflow_dispatch:

schedule:
# run Thursday 4:30 AM UTC
- cron: '30 4 * * 4'
jobs:
test:
runs-on: ubuntu-20.04
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved

env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}

steps:
- name: Checkout source code
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved
uses: actions/checkout@v4

- name: Set up Docker Compose
run: |
sudo apt-get update
sudo apt-get -y install docker-compose

- name: Build Containers
run: |
cp docker/env.example .env
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved
echo "COMPOSE_PROJECT_NAME=pecan" >> .env
echo "PECAN_VERSION=develop" >> .env
echo "UID=$(id -u)" >> .env
echo "GID=$(id -g)" >> .env

docker-compose up -d postgres
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved
docker run --rm --network pecan_pecan pecan/db
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved

docker-compose run bety user guestuser guestuser "Guest User" guestuser@example.com 4 4
docker-compose run bety user carya illinois "Carya Demo User" carya@example.com 1 1
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved

docker run --rm --network pecan_pecan --volume pecan_pecan:/data --env FQDN=docker pecan/data:develop
docker run --rm --network pecan_pecan --volume pecan_pecan:/data pecan/data:develop chown -R "$(id -u).$(id -g)" /data

docker run --user="$(id -u)" --rm --network pecan_pecan --volume pecan_pecan:/data --env FQDN=docker pecan/data:develop
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved

- name: Run required containers
run: |
docker-compose up pecan bety sipnet rstudio -d
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved

- name: Check if containers are running # This will be updated soon to a loop
run: |
docker ps
sleep 50
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved

- name: Run SIPNET
run: |
docker exec --workdir /pecan/tests pecan-rstudio-1 R CMD ../web/workflow.R --settings test.xml
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ services:
security_opt:
- no-new-privileges:true
ports:
- "${TRAEFIK_HTTP_PORT-80}:80"
- "${TRAEFIK_HTTP_PORT:-80}:80"
volumes:
- "traefik:/config"
- "/var/run/docker.sock:/var/run/docker.sock:ro"
Expand Down
68 changes: 68 additions & 0 deletions tests/test.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
<?xml version="1.0"?><pecan>
<outdir>/data/workflows/PEcAn_99000000021</outdir>
meetagrawal09 marked this conversation as resolved.
Show resolved Hide resolved
<database>
<bety>
<user>bety</user>
<password>bety</password>
<host>postgres</host>
<port>5432</port>
<dbname>bety</dbname>
<driver>PostgreSQL</driver>
<write>true</write>
</bety>
<dbfiles>/data/dbfiles</dbfiles>
</database>
<pfts>
<pft>
<name>semiarid.grassland</name>
</pft>
</pfts>
<meta.analysis>
<iter>3000</iter>
<random.effects>
<on>FALSE</on>
<use_ghs>TRUE</use_ghs>
</random.effects>
</meta.analysis>
<ensemble>
<size>1</size>
<variable>NPP</variable>
<samplingspace>
<parameters>
<method>uniform</method>
</parameters>
<met>
<method>sampling</method>
</met>
</samplingspace>
</ensemble>
<model>
<id>1000000014</id>
</model>
<workflow>
<id>99000000029</id>
</workflow>
<run>
<site>
<id>2000001297</id>
<met.start>2004/01/01</met.start>
<met.end>2004/12/31</met.end>
</site>
<inputs>
<met>
<source>CRUNCEP</source>
<output>SIPNET</output>
<product/>
</met>
</inputs>
<start.date>2004/01/01</start.date>
<end.date>2004/12/31</end.date>
</run>
<host>
<name>localhost</name>
<rabbitmq>
<uri>amqp://carya:illinois@rabbitmq/%2F</uri>
<queue>SIPNET_r136</queue>
</rabbitmq>
</host>
</pecan>
Loading