Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add github workflow tests for FATES luh2 data tool #1059

Draft
wants to merge 56 commits into
base: main
Choose a base branch
from
Draft

Add github workflow tests for FATES luh2 data tool #1059

wants to merge 56 commits into from

Commits on Mar 1, 2023

  1. Add luh2 module and conda environment yaml file

    @glemieux
    glemieux committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    a453cba View commit details
    Browse the repository at this point in the history

  2. Add function to update luh2 time units

    @glemieux
    glemieux committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    fda4aff View commit details
    Browse the repository at this point in the history

  3. Update conda environment for luh2 tools

    @glemieux
    glemieux committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    c7871a9 View commit details
    Browse the repository at this point in the history

Commits on Mar 2, 2023

  1. Adding functionality notes

    @glemieux
    glemieux committed Mar 2, 2023
    Configuration menu
    Copy the full SHA
    f464b7b View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2023

  1. Start bringing in Charlie's xesmf code

    @glemieux
    glemieux committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    202b09c View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2023

  1. Update yml environment file

    @glemieux
    glemieux committed Mar 13, 2023
    Configuration menu
    Copy the full SHA
    6ef45bb View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2023

  1. Continue converting Charlie's script

    @glemieux
    glemieux committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    4c9a47f View commit details
    Browse the repository at this point in the history

  2. Update notes

    @glemieux
    glemieux committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    134ed79 View commit details
    Browse the repository at this point in the history

Commits on Mar 16, 2023

  1. Update comments

    @glemieux
    glemieux committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    da4a06d View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2023

  1. Renamed luh2 module and added primary script

    @glemieux
    glemieux committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    2b7364e View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2023

  1. Moving script into module function

    @glemieux
    glemieux committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    6a32ff0 View commit details
    Browse the repository at this point in the history

  2. Intermediate improvements

    @glemieux
    glemieux committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    d3e347f View commit details
    Browse the repository at this point in the history

  3. Fixup

    @glemieux
    glemieux committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    f334f22 View commit details
    Browse the repository at this point in the history

Commits on Apr 6, 2023

  1. Update to xarray 0.19.0 

    This incorporates an update to handle a pandas deprication issue when
using cftime objects.  See pydata/xarray#5359
    @glemieux
    glemieux committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    f8fd35f View commit details
    Browse the repository at this point in the history

  2. Truncate to the yml file and update prep function 

    Having too many of the dependencies in the yml file was causing issue
with the dependency order and limiting xesmf to an older version.
    @glemieux
    glemieux committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    0490748 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2023

  1. more intermediate updates

    @glemieux
    glemieux committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    91ef322 View commit details
    Browse the repository at this point in the history

  2. Add command line argument parsing

    @glemieux
    glemieux committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    5b65edd View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2023

  1. continuing to add updates and debug

    @glemieux
    glemieux committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    05f52dd View commit details
    Browse the repository at this point in the history

Commits on May 1, 2023

  1. add year variable as copy of time dimension 

    This is necessary due to a ncd_io read requirement set in dynsubgrid.
    @glemieux
    glemieux committed May 1, 2023
    Configuration menu
    Copy the full SHA
    bf07950 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2023

  1. converting variables to snake case

    @glemieux
    glemieux committed May 4, 2023
    Configuration menu
    Copy the full SHA
    d879aea View commit details
    Browse the repository at this point in the history

  2. adjust regridding methods

    @glemieux
    glemieux committed May 4, 2023
    Configuration menu
    Copy the full SHA
    848bbea View commit details
    Browse the repository at this point in the history

Commits on May 5, 2023

  1. various fixes and improvements

    @glemieux
    glemieux committed May 5, 2023
    Configuration menu
    Copy the full SHA
    c2e4e24 View commit details
    Browse the repository at this point in the history

  2. Separated luh2 module and luh2 call script

    @glemieux
    glemieux committed May 5, 2023
    Configuration menu
    Copy the full SHA
    99eac94 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2023

  1. More updates

    @glemieux
    glemieux committed May 8, 2023
    Configuration menu
    Copy the full SHA
    b04fd2e View commit details
    Browse the repository at this point in the history

  2. start working on allowing read in of regridder file

    @glemieux
    glemieux committed May 8, 2023
    Configuration menu
    Copy the full SHA
    472eaa6 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2023

  1. Fix regridding loop

    @glemieux
    glemieux committed May 9, 2023
    Configuration menu
    Copy the full SHA
    2e1c736 View commit details
    Browse the repository at this point in the history

  2. Remove unnecessary xarray import call

    @glemieux
    glemieux committed May 9, 2023
    Configuration menu
    Copy the full SHA
    92a57d0 View commit details
    Browse the repository at this point in the history

Commits on May 12, 2023

  1. Adding state correction function

    @glemieux
    glemieux committed May 12, 2023
    Configuration menu
    Copy the full SHA
    5c72c2f View commit details
    Browse the repository at this point in the history

Commits on May 13, 2023

  1. Add handling of regridder argument option

    @glemieux
    glemieux committed May 13, 2023
    Configuration menu
    Copy the full SHA
    0b86cd2 View commit details
    Browse the repository at this point in the history

  2. switch the importdata and prepdataset functions

    @glemieux
    glemieux committed May 13, 2023
    Configuration menu
    Copy the full SHA
    430bd03 View commit details
    Browse the repository at this point in the history

  3. Correct some issues

    @glemieux
    glemieux committed May 13, 2023
    Configuration menu
    Copy the full SHA
    3a31c2a View commit details
    Browse the repository at this point in the history

  4. adding type checks

    @glemieux
    glemieux committed May 13, 2023
    Configuration menu
    Copy the full SHA
    9af66f0 View commit details
    Browse the repository at this point in the history

  5. adding check against the time variable for the input dataset

    @glemieux
    glemieux committed May 13, 2023
    Configuration menu
    Copy the full SHA
    23dabb7 View commit details
    Browse the repository at this point in the history

  6. fix import function

    @glemieux
    glemieux committed May 13, 2023
    Configuration menu
    Copy the full SHA
    61306a0 View commit details
    Browse the repository at this point in the history

  7. fix argument call

    @glemieux
    glemieux committed May 13, 2023
    Configuration menu
    Copy the full SHA
    4e830a8 View commit details
    Browse the repository at this point in the history

  8. update check function outputs

    @glemieux
    glemieux committed May 13, 2023
    Configuration menu
    Copy the full SHA
    e4b43ac View commit details
    Browse the repository at this point in the history

Commits on May 16, 2023

  1. Add bash script

    @glemieux
    glemieux committed May 16, 2023
    Configuration menu
    Copy the full SHA
    f22540d View commit details
    Browse the repository at this point in the history

Commits on May 17, 2023

  1. refactoring luh2 module 

    This refactor is to account for a misunderstanding in the way that
the reuse of a saved regridder weights file is to be applied.
    @glemieux
    glemieux committed May 17, 2023
    Configuration menu
    Copy the full SHA
    eb79dd8 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2023

  1. Add merge flag to import data 

    This avoids prepping the data if its already been prepped
    @glemieux
    glemieux committed May 18, 2023
    Configuration menu
    Copy the full SHA
    1941e66 View commit details
    Browse the repository at this point in the history

  2. fix order of operations

    @glemieux
    glemieux committed May 18, 2023
    Configuration menu
    Copy the full SHA
    1cee030 View commit details
    Browse the repository at this point in the history

  3. add commands to remove intermediate file copies 

    This helps mitigate loss of storage space
    @glemieux
    glemieux committed May 18, 2023
    Configuration menu
    Copy the full SHA
    4eadfb7 View commit details
    Browse the repository at this point in the history

  4. move all luh2 code into named folder

    @glemieux
    glemieux committed May 18, 2023
    Configuration menu
    Copy the full SHA
    e6278f6 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2023

  1. convert YEAR from cftime object to number

    @glemieux
    glemieux committed May 23, 2023
    Configuration menu
    Copy the full SHA
    28bcafc View commit details
    Browse the repository at this point in the history

Commits on May 31, 2023

  1. bugfixes to reduce memory usage and interpret cftime reference point

    @ckoven
    ckoven committed May 31, 2023
    Configuration menu
    Copy the full SHA
    a178f9d View commit details
    Browse the repository at this point in the history

  2. Merge pull request #28 from ckoven/fates-luh2_data 

    bugfixes to reduce memory usage and interpret cftime reference point
    @glemieux
    glemieux committed May 31, 2023
    Configuration menu
    Copy the full SHA
    846f7d3 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2023

  1. implement decode_times=False

    @glemieux
    glemieux committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    d652c21 View commit details
    Browse the repository at this point in the history

  2. remove call to attribute update in shell script

    @glemieux
    glemieux committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    14153c5 View commit details
    Browse the repository at this point in the history

  3. reworking luh2 shell script call

    @glemieux
    glemieux committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    35dee92 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2023

  1. fix choice range and add shell script input options

    @glemieux
    glemieux committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    ec19393 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. Add comments and remove unnecessary dependence to yaml file

    @glemieux
    glemieux committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    67b8d01 View commit details
    Browse the repository at this point in the history

  2. add luh2 readme

    @glemieux
    glemieux committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    63124fd View commit details
    Browse the repository at this point in the history

  3. remove old comments

    @glemieux
    glemieux committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    e44bc90 View commit details
    Browse the repository at this point in the history

  4. Add system exit if unrecognized file type provided as argument

    @glemieux
    glemieux committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    4615b8a View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2023

  1. update luh2 tool argparse options and time handling 

    Adding 'default' and 'required' option as necessary to the argparse.
This helps remove some of the luh2mod code that was previously
handling default options.

This also adds code to handle non-historical luh2 data file input
arguments.  Prior to this, it was assumed that only historical
data files starting at 0850-01-01 were to be used.  Now this
can handle luh2 scenario files.  It also adds time bounds error handling.
    @glemieux
    glemieux committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    8cbb7ab View commit details
    Browse the repository at this point in the history

  2. update the usage description in luh2.py

    @glemieux
    glemieux committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    9cbfe4f View commit details
    Browse the repository at this point in the history

  3. Add template workflow for luh2 tools testing 

    This is adapted from the "Python Package using Anaconda".
This will need additional updates.
    @glemieux
    glemieux committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    37391bb View commit details
    Browse the repository at this point in the history