Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(generate_classes): create a command-line interface #1912

Merged
merged 1 commit into from
Aug 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -209,12 +209,12 @@ jobs:

- name: Update FloPy packages
if: runner.os != 'Windows'
run: python -c 'import flopy; flopy.mf6.utils.generate_classes(ref="develop", backup=False)'
run: python -m flopy.mf6.utils.generate_classes --ref develop --no-backup

- name: Update FloPy packages
if: runner.os == 'Windows'
shell: bash -l {0}
run: python -c 'import flopy; flopy.mf6.utils.generate_classes(ref="develop", backup=False)'
run: python -m flopy.mf6.utils.generate_classes --ref develop --no-backup

- name: Run tests
if: runner.os != 'Windows'
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,12 +86,12 @@ jobs:

- name: Update FloPy packages
if: runner.os != 'Windows'
run: python -c 'import flopy; flopy.mf6.utils.generate_classes(ref="develop", backup=False)'
run: python -m flopy.mf6.utils.generate_classes --ref develop --no-backup

- name: Update FloPy packages
if: runner.os == 'Windows'
shell: bash -l {0}
run: python -c 'import flopy; flopy.mf6.utils.generate_classes(ref="develop", backup=False)'
run: python -m flopy.mf6.utils.generate_classes --ref develop --no-backup

- name: Run example tests
if: runner.os != 'Windows'
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/regression.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,12 +70,12 @@ jobs:

- name: Update FloPy packages
if: runner.os != 'Windows'
run: python -c 'import flopy; flopy.mf6.utils.generate_classes(ref="develop", backup=False)'
run: python -m flopy.mf6.utils.generate_classes --ref develop --no-backup

- name: Update FloPy packages
if: runner.os == 'Windows'
shell: bash -l {0}
run: python -c 'import flopy; flopy.mf6.utils.generate_classes(ref="develop", backup=False)'
run: python -m flopy.mf6.utils.generate_classes --ref develop --no-backup

- name: Run regression tests
if: runner.os != 'Windows'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
echo "version=${ver#"v"}" >> $GITHUB_OUTPUT

- name: Update FloPy packages
run: python -c 'import flopy; flopy.mf6.utils.generate_classes(ref="master", backup=False)'
run: python -m flopy.mf6.utils.generate_classes --ref master --no-backup

- name: Lint Python files
run: python scripts/pull_request_prepare.py
Expand Down
9 changes: 2 additions & 7 deletions autotest/test_generate_classes.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,13 +100,8 @@ def test_generate_classes_from_github_refs(
ref = spl[2]
pprint(
virtualenv.run(
"python -c 'from flopy.mf6.utils import generate_classes; generate_classes(owner=\""
+ owner
+ '", repo="'
+ repo
+ f'", ref="'
+ ref
+ "\", backup=False)'"
"python -m flopy.mf6.utils.generate_classes "
f"--owner {owner} --repo {repo} ref {ref} --no-backup"
)
)

Expand Down
62 changes: 46 additions & 16 deletions docs/generate_classes.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,8 @@ MODFLOW 6 input continues to evolve as new models, packages, and options are dev

The FloPy classes for MODFLOW 6 are largely generated by a utility which converts DFN files in a modflow6 repository on GitHub or on the local machine into Python source files in your local FloPy install. For instance (output much abbreviated):

```python
>>> from flopy.mf6.utils import generate_classes
>>> generate_classes()
```bash
$ python -m flopy.mf6.utils.generate_classes



Expand Down Expand Up @@ -39,27 +38,58 @@ LIST OF FILES IN C:\Users\***\flopy\flopy\mf6\modflow
2 - mfgwf.py
...
```

The `generate_classes()` function has several optional parameters.

Similar functionality is available within Python, e.g.:
```python
# use the develop branch instead
generate_classes(ref="develop")
>>> from flopy.mf6.utils import generate_classes
>>> generate_classes()
```

# use a fork of modflow6
generate_classes(owner="your-username", ref="your-branch")
The `generate_classes()` function has several optional parameters.

# maybe your fork has a different name
generate_classes(owner="your-username", repo="your-modflow6", ref="your-branch")
```bash
$ python -m flopy.mf6.utils.generate_classes -h
usage: generate_classes.py [-h] [--owner OWNER] [--repo REPO] [--ref REF]
[--dfnpath DFNPATH] [--no-backup]

Generate the MODFLOW 6 flopy classes using definition files from the MODFLOW 6
GitHub repository or a set of definition files in a folder provided by the
user.

options:
-h, --help show this help message and exit
--owner OWNER GitHub repository owner; default is 'MODFLOW-USGS'.
--repo REPO Name of GitHub repository; default is 'modflow6'.
--ref REF Branch name, tag, or commit hash to use to update the
definition; default is 'master'.
--dfnpath DFNPATH Path to a definition file folder that will be used to
generate the MODFLOW 6 classes.
--no-backup Set to disable backup. Default behavior is to keep a
backup of the definition files in dfn_backup with a date
and timestamp from when the definition files were
replaced.
```

# local copy of the repo
generate_classes(dfnpath="../your/dfn/path")
For example, use the develop branch instead:
```bash
$ python -m flopy.mf6.utils.generate_classes --ref develop
```
use a fork of modflow6:
```bash
$ python -m flopy.mf6.utils.generate_classes --owner your-username --ref your-branch
```
maybe your fork has a different name:
```bash
$ python -m flopy.mf6.utils.generate_classes --owner your-username --repo your-modflow6 --ref your-branch
```
local copy of the repo:
```bash
$ python -m flopy.mf6.utils.generate_classes --dfnpath ../your/dfn/path
```

Branch names, commit hashes, or tags may be provided to `ref`.

By default, a backup is made of FloPy's package classes before rewriting them. To disable backups, use `backup=False`.
By default, a backup is made of FloPy's package classes before rewriting them. To disable backups, use `--no-backup` from command-line, or `backup=False` with the Python function.

## Testing class generation

Tests for the `generate_classes()` utility are located in `test_generate_classes.py`. The tests depend on [`pytest-virtualenv`](https://pypi.org/project/pytest-virtualenv/) and will be skipped if run in parallel without the `--dist loadfile` option for `pytest-xdist`.
Tests for the `generate_classes()` utility are located in `test_generate_classes.py`. The tests depend on [`pytest-virtualenv`](https://pypi.org/project/pytest-virtualenv/) and will be skipped if run in parallel without the `--dist loadfile` option for `pytest-xdist`.
2 changes: 1 addition & 1 deletion docs/make_release.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ As described above, making a release manually involves the following steps:

- Run `python scripts/update_version.py -v <semver>` to update the version number stored in `version.txt` and `flopy/version.py`. For an approved release use the `--approve` flag.

- Update MODFLOW 6 dfn files in the repository and MODFLOW 6 package classes by running `python -c 'import flopy; flopy.mf6.utils.generate_classes(ref="master", backup=False)'`
- Update MODFLOW 6 dfn files in the repository and MODFLOW 6 package classes by running `python -m flopy.mf6.utils.generate_classes --ref master --no-backup`

- Run `isort` and `black` on the `flopy` module. This can be achieved by running `python scripts/pull_request_prepare.py` from the project root. The commands `isort .` and `black .` can also be run individually instead.

Expand Down
92 changes: 74 additions & 18 deletions flopy/mf6/utils/generate_classes.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@
flopypth = os.path.abspath(flopypth)
protected_dfns = ["flopy.dfn"]

default_owner = "MODFLOW-USGS"
default_repo = "modflow6"


def delete_files(files, pth, allow_failure=False, exclude=None):
if exclude is None:
Expand Down Expand Up @@ -51,13 +54,12 @@ def list_files(pth, exts=["py"]):
def download_dfn(owner, branch, new_dfn_pth):
try:
from modflow_devtools.download import download_and_unzip
except:
msg = (
"Error. The modflow-devtools package must be installed in order to "
"generate the MODFLOW 6 classes. modflow-devtools can be installed using "
"pip install modflow-devtools. Stopping."
except ImportError:
raise ImportError(
"The modflow-devtools package must be installed in order to "
"generate the MODFLOW 6 classes. This can be with:\n"
" pip install modflow-devtools"
)
print(msg)

mf6url = "https://github.com/{}/modflow6/archive/{}.zip"
mf6url = mf6url.format(owner, branch)
Expand Down Expand Up @@ -110,10 +112,10 @@ def delete_mf6_classes():


def generate_classes(
owner="MODFLOW-USGS",
repo="modflow6",
branch="master",
ref=None,
owner=default_owner,
repo=default_repo,
branch=None,
ref="master",
dfnpath=None,
backup=True,
):
Expand All @@ -124,27 +126,27 @@ def generate_classes(

Parameters
----------
owner : str
owner : str, default "MODFLOW-USGS"
Owner of the MODFLOW 6 repository to use to update the definition
files and generate the MODFLOW 6 classes. Default is MODFLOW-USGS.
repo : str
files and generate the MODFLOW 6 classes.
repo : str, default "modflow6"
Name of the MODFLOW 6 repository to use to update the definition.
branch : str
branch : str, optional
Branch name of the MODFLOW 6 repository to use to update the
definition files and generate the MODFLOW 6 classes. Default is master.
definition files and generate the MODFLOW 6 classes.

.. deprecated:: 3.5.0
Use ref instead.
ref : str
ref : str, default "master"
Branch name, tag, or commit hash to use to update the definition.
dfnpath : str
Path to a definition file folder that will be used to generate the
MODFLOW 6 classes. Default is none, which means that the branch
will be used instead. dfnpath will take precedence over branch
if dfnpath is specified.
backup : bool
backup : bool, default True
Keep a backup of the definition files in dfn_backup with a date and
time stamp from when the definition files were replaced.
timestamp from when the definition files were replaced.

"""

Expand Down Expand Up @@ -191,3 +193,57 @@ def generate_classes(
print(" Create mf6 classes using the downloaded definition files.")
create_packages()
list_files(os.path.join(flopypth, "mf6", "modflow"))


def cli_main():
"""Command-line interface for generate_classes()."""
import argparse
import sys

parser = argparse.ArgumentParser(
description=generate_classes.__doc__.split("\n\n")[0],
)

parser.add_argument(
"--owner",
type=str,
default=default_owner,
help=f"GitHub repository owner; default is '{default_owner}'.",
)
parser.add_argument(
"--repo",
default=default_repo,
help=f"Name of GitHub repository; default is '{default_repo}'.",
)
parser.add_argument(
"--ref",
default="master",
help="Branch name, tag, or commit hash to use to update the "
"definition; default is 'master'.",
)
parser.add_argument(
"--dfnpath",
help="Path to a definition file folder that will be used to generate "
"the MODFLOW 6 classes.",
)
parser.add_argument(
"--no-backup",
action="store_true",
help="Set to disable backup. "
"Default behavior is to keep a backup of the definition files in "
"dfn_backup with a date and timestamp from when the definition "
"files were replaced.",
)

args = vars(parser.parse_args())
# Handle flipped logic
args["backup"] = not args.pop("no_backup")
try:
generate_classes(**args)
except (EOFError, KeyboardInterrupt):
sys.exit(f" cancelling '{sys.argv[0]}'")


if __name__ == "__main__":
"""Run command-line with: python -m flopy.mf6.utils.generate_classes"""
cli_main()