Skip to content

Commit

Permalink
Merge pull request #47 from NREL/breaks_style
Browse files Browse the repository at this point in the history
Breaks Scheme for make-maps
  • Loading branch information
mjgleason authored Oct 4, 2023
2 parents 9742821 + 25ad0e1 commit 951b7ed
Show file tree
Hide file tree
Showing 13 changed files with 403 additions and 97 deletions.
2 changes: 0 additions & 2 deletions .github/workflows/pull_request_tests_unix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,6 @@ jobs:
fetch-depth: 1

- uses: nanasess/setup-chromedriver@v2
with:
chromedriver-version: '115.0.5790.170'
- name: Install Chrome Driver
run: |
export DISPLAY=:99
Expand Down
54 changes: 35 additions & 19 deletions USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,28 +53,44 @@ The `make-maps` command can be used to generate a small set of standardized, rep

This command can be run according to the following usage:
```commandline
Usage: make-maps [OPTIONS]
Usage: reView-tools make-maps [OPTIONS]
Generates standardized, presentation-quality maps for the input supply
curve, including maps for each of the following attributes: Capacity
(capacity), All-in LCOE (total_lcoe), Project LCOE (mean_lcoe), LCOT (lcot),
Capacity Density (derived column) [wind only]
Generates standardized, presentation-quality maps for the input supply curve, including maps for
each of the following attributes: Capacity (capacity), All-in LCOE (total_lcoe), Project LCOE
(mean_lcoe), LCOT (lcot), Capacity Density (derived column) [wind only]
Options:
-i, --supply_curve_csv FILE Path to supply curve CSV file. [required]
-t, --tech [wind|solar] Technology choice for ordinances to export.
Valid options are: ['wind', 'solar'].
[required]
-o, --out_folder DIRECTORY Path to output folder for maps. [required]
-b, --boundaries FILE Path to vector dataset with the boundaries to
map. Default is to use state boundaries for
CONUS from Natural Earth (1:50m scale), which
is suitable for CONUS supply curves. For other
region, it is recommended to provide a more
appropriate boundaries dataset.
-d, --dpi INTEGER RANGE Dots-per-inch (DPI) for output images. Default
is 600. [x>=0]
--help Show this message and exit.
-i, --supply_curve_csv FILE Path to supply curve CSV file. [required]
-S, --breaks-scheme TEXT The format for this option is either 'wind' or 'solar', for the
hard-coded breaks for those technologies, or '<classifier-
name>:<classifier-kwargs>' where <classifier-name> is one of the
valid classifiers from the mapclassify package (see
https://pysal.org/mapclassify/api.html#classifiers) and
<classifier-kwargs> is an optional set of keyword arguments to
pass to the classifier function, formatted as a JSON. So, a valid
input would be 'equalinterval:{"k": 10}' (this would produce 10
equal interval breaks). Note that this should all be entered as a
single string, wrapped in single quotes. Alternatively the user
can specify just 'equalinterval' without the kwargs JSON for the
equal interval classifier to be used with its default 5 bins (in
this case, wrapping the string in single quotes is optional) The
--breaks-scheme option must be specified unless the legacy --tech
option is used instead.
-t, --tech TEXT Alias for --breaks-scheme. For backwards compatibility only.
-o, --out_folder DIRECTORY Path to output folder for maps. [required]
-b, --boundaries FILE Path to vector dataset with the boundaries to map. Default is to
use state boundaries for CONUS from Natural Earth (1:50m scale),
which is suitable for CONUS supply curves. For other region, it is
recommended to provide a more appropriate boundaries dataset. The
input vector dataset can be in CRS.
-K, --keep-zero Keep zero capacity supply curve project sites. These sites are
dropped by default.
-d, --dpi INTEGER RANGE Dots-per-inch (DPI) for output images. Default is 600. [x>=0]
-F, --out-format [png|pdf|svg|jpg]
Output format for images. Default is ``png`` Valid options are:
['png', 'pdf', 'svg', 'jpg'].
-D, --drop-legend Drop legend from map. Legend is shown by default.
--help Show this message and exit.
```

This command intentionally limits the options available to the user because it is meant to produce standard maps that are commonly desired for any supply curve. The main changes that the user can make are to change the DPI of the output image (e.g., for less detaild/smaller image file sizes, set to 300) and to provide a custom `--boundaries` vector dataset. The latter option merits some additional explanation.
Expand Down
146 changes: 135 additions & 11 deletions reView/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import numpy as np
import matplotlib.pyplot as plt
import tqdm
import mapclassify as mc

from reView.utils.bespoke import batch_unpack_from_supply_curve
from reView.utils import characterizations, plots
Expand All @@ -22,8 +23,8 @@
logger = logging.getLogger(__name__)

CONTEXT_SETTINGS = {
"max_content_width": 9999,
"terminal_width": 9999
"max_content_width": 100,
"terminal_width": 100
}
TECH_CHOICES = ["wind", "solar"]
DEFAULT_BOUNDARIES = Path(REVIEW_DATA_DIR).joinpath(
Expand Down Expand Up @@ -140,16 +141,103 @@ def unpack_characterizations(
char_df.to_csv(out_csv, header=True, index=False, mode="x")


def validate_breaks_scheme(ctx, param, value):
# pylint: disable=unused-argument
"""
Custom validation for --break-scheme/--techs input to make-maps command.
Checks that the input value is either one of the valid technologies,
None, or a string specifying a mapclassifier classifier and (optionally)
its keyword arguments, delimited by a colon (e.g.,
'equalinterval:{"k":10}'.)
Parameters
----------
ctx : click.core.Context
Unused
param : click.core.Option
Unused
value : [str, None]
Value of the input parameter
Returns
-------
[str, None, tuple]
Returns one of the following:
- a string specifying a technology name
- None type (if the input value was not specified)
- a tuple of the format (str, dict), where the string is the name
of a mapclassify classifier and the dictionary are the keyword
arguments to be passed to that classfier.
Raises
------
click.BadParameter
A BadParameter exception will be raised if either of the following
cases are encountered:
- an invalid classifier name is specified
- the kwargs do not appear to be valid JSON
"""


if value in TECH_CHOICES or value is None:
return value

classifier_inputs = value.split(":", maxsplit=1)
classifier = classifier_inputs[0].lower()
if classifier not in [c.lower() for c in mc.CLASSIFIERS]:
raise click.BadParameter(
f"Classifier {classifier} not recognized as one of the valid "
f"options: {mc.CLASSIFIERS}."
)

if len(classifier_inputs) == 1:
classifier_kwargs = {}
elif len(classifier_inputs) == 2:
try:
classifier_kwargs = json.loads(classifier_inputs[1])
except json.decoder.JSONDecodeError as e:
raise click.BadParameter(
"Keyword arguments for classifier must be formated as valid "
"JSON."
) from e

return classifier, classifier_kwargs


@main.command()
@click.option('--supply_curve_csv', '-i', required=True,
type=click.Path(exists=True, dir_okay=False, file_okay=True),
help='Path to supply curve CSV file.')
@click.option("--breaks-scheme",
"-S",
required=False,
type=click.STRING,
callback=validate_breaks_scheme,
help=("The format for this option is either 'wind' or 'solar', "
"for the hard-coded breaks for those technologies, or "
"'<classifier-name>:<classifier-kwargs>' where "
"<classifier-name> is one of the valid classifiers from "
"the mapclassify package "
"(see https://pysal.org/mapclassify/api.html#classifiers) "
"and <classifier-kwargs> is an optional set of keyword "
"arguments to pass to the classifier function, formatted "
"as a JSON. So, a valid input would be "
"'equalinterval:{\"k\": 10}' (this would produce 10 equal "
"interval breaks). Note that this should all be entered "
"as a single string, wrapped in single quotes. "
"Alternatively the user can specify just 'equalinterval' "
"without the kwargs JSON for the equal interval "
"classifier to be used with its default 5 bins (in this "
"case, wrapping the string in single quotes is optional) "
"The --breaks-scheme option must be specified unless the "
"legacy --tech option is used instead."))
@click.option("--tech",
"-t",
required=True,
type=click.Choice(TECH_CHOICES, case_sensitive=False),
help="Technology choice for ordinances to export. "
f"Valid options are: {TECH_CHOICES}.")
required=False,
type=click.STRING,
callback=validate_breaks_scheme,
help="Alias for --breaks-scheme. For backwards compatibility "
"only.")
@click.option('--out_folder', '-o', required=True,
type=click.Path(exists=False, dir_okay=True, file_okay=False),
help='Path to output folder for maps.')
Expand Down Expand Up @@ -183,8 +271,8 @@ def unpack_characterizations(
is_flag=True,
help='Drop legend from map. Legend is shown by default.')
def make_maps(
supply_curve_csv, tech, out_folder, boundaries, keep_zero, dpi, out_format,
drop_legend
supply_curve_csv, breaks_scheme, tech, out_folder, boundaries, keep_zero,
dpi, out_format, drop_legend
):
"""
Generates standardized, presentation-quality maps for the input supply
Expand All @@ -193,6 +281,18 @@ def make_maps(
LCOT (lcot), Capacity Density (derived column) [wind only]
"""

if tech is None and breaks_scheme is None:
raise click.MissingParameter(
"Either --breaks-scheme or --tech must be specified."
)
if tech is not None and breaks_scheme is not None:
warnings.warn(
"Both --breaks-scheme and --tech were specified: "
"input for --tech will be ignored"
)
if tech is not None and breaks_scheme is None:
breaks_scheme = tech

out_path = Path(out_folder)
out_path.mkdir(exist_ok=True, parents=False)

Expand Down Expand Up @@ -254,9 +354,21 @@ def make_maps(
"breaks": [5, 10, 25, 50, 100, 120],
"cmap": "BuPu",
"legend_title": "Developable Area (sq km)"
},
cap_col: {
"breaks": None,
"cmap": 'PuRd',
"legend_title": "Capacity (MW)"
},
"capacity_density": {
"breaks": None,
"cmap": 'PuRd',
"legend_title": "Capacity Density (MW/sq km)"
}
}
if tech == "solar":

if breaks_scheme == "solar":
out_suffix = breaks_scheme
ac_cap_col = find_capacity_column(
supply_curve_df,
cap_col_candidates=["capacity_ac", "capacity_mw_ac"]
Expand All @@ -278,7 +390,8 @@ def make_maps(
"legend_title": "Capacity Density (MW/sq km)"
}
})
elif tech == "wind":
elif breaks_scheme == "wind":
out_suffix = breaks_scheme
map_vars.update({
cap_col: {
"breaks": [60, 120, 180, 240, 275],
Expand All @@ -291,6 +404,17 @@ def make_maps(
"legend_title": "Capacity Density (MW/sq km)"
}
})
else:
classifier, classifier_kwargs = breaks_scheme
out_suffix = classifier
# pylint: disable=consider-using-dict-items, consider-iterating-dictionary
for map_var in map_vars.keys():
scheme = mc.classify(
supply_curve_gdf[map_var], classifier, **classifier_kwargs
)
breaks = scheme.bins
map_vars[map_var]["breaks"] = breaks.tolist()[0:-1]

for map_var, map_settings in tqdm.tqdm(map_vars.items()):
g = plots.map_geodataframe_column(
supply_curve_gdf,
Expand All @@ -316,7 +440,7 @@ def make_maps(
g.figure.set_figwidth(fig_height * bbox.width / bbox.height)
plt.tight_layout(pad=0.1)

out_image_name = f"{map_var}_{tech}.{out_format}"
out_image_name = f"{map_var}_{out_suffix}.{out_format}"
out_image_path = out_path.joinpath(out_image_name)
g.figure.savefig(out_image_path, dpi=dpi, transparent=True)
plt.close(g.figure)
Expand Down
8 changes: 8 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@
from reView import TEST_DATA_DIR
from reView.utils.functions import load_project_configs

from tests import helper


@pytest.fixture
def data_dir_test():
Expand Down Expand Up @@ -302,6 +304,12 @@ def histogram_plot_capacity_mw_5bins():
return contents


@pytest.fixture
def compare_images_approx():
"""Exposes the compare_images_approx function as a fixture"""
return helper.compare_images_approx


def pytest_setup_options():
"""Recommended setup based on https://dash.plotly.com/testing."""
options = Options()
Expand Down
Binary file added tests/data/plots/area_sq_km_equalinterval.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/data/plots/capacity_mw_equalinterval.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/data/plots/lcot_equalinterval.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/data/plots/mean_lcoe_equalinterval.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/data/plots/total_lcoe_equalinterval.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
49 changes: 49 additions & 0 deletions tests/helper.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# -*- coding: utf-8 -*-
"""Helper functions for tests"""
import PIL
import imagehash
import numpy as np


def compare_images_approx(
image_1_path, image_2_path, hash_size=12, max_diff_pct=0.25
):
"""
Check if two images match approximately.
Parameters
----------
image_1_path : pathlib.Path
File path to first image.
image_2_path : pathlib.Path
File path to first image.
hash_size : int, optional
Size of the image hashes that will be used for image comparison,
by default 12. Increase to make the check more precise, decrease to
make it more approximate.
max_diff_pct : float, optional
Tolerance for the amount of difference allowed, by default 0.05 (= 5%).
Increase to allow for a larger delta between the image hashes, decrease
to make the check stricter and require a smaller delta between the
image hashes.
Returns
-------
bool
Returns true if the images match approximately, false if not.
"""

expected_hash = imagehash.phash(
PIL.Image.open(image_1_path), hash_size=hash_size
)
out_hash = imagehash.phash(
PIL.Image.open(image_2_path), hash_size=hash_size
)

max_diff_bits = int(np.ceil(hash_size * max_diff_pct))

diff = expected_hash - out_hash
matches = diff <= max_diff_bits
pct_diff = float(diff) / hash_size

return matches, pct_diff
Loading

0 comments on commit 951b7ed

Please sign in to comment.