Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NetCDF4/HDF5 support, including support for compressed NetCDF trajectories. #999

Merged
merged 227 commits into from
Nov 17, 2022

Conversation

drroe
Copy link
Contributor

@drroe drroe commented Nov 16, 2022

Version 6.16.0.

This PR adds support for the NetCDF4/HDF5 format in addition to the current NetCDF 3 (classic) format currently used by Amber. The main advantage of this is it makes it possible to natively compress NetCDF trajectories via the underlying HDF5 framework. There are currently two options for compressing NetCDF trajectories; lossless, and lossy; the latter uses integer quantization to improve compression ratios and is explained in an upcoming paper to be published in Protein Science.

  • Support NetCDF4/HDF5 formats (only serial currently) - hdf5 keyword for NetCDF trajout.
  • Support compression and lossy compression of NetCDF4/HDF5 formats - compress/icompress keywords for NetCDF trajout.
  • Support reading (not writing) MDtraj H5 and MDAnalysis H5MD trajectories.
  • Add nokinetic keyword to the energy action to suppress kinetic energy calculation when velocity information is present.
  • Can now read Amber energy files (amberene).
  • Add prec {single|double} keyword to loadcrd command to specify saving coordinates as single (COORDS) or double (FRAMES) precision.
  • Add bondparm keyword to the change command for modifying bond parameters.
  • Add concatenation of string data sets with dataset cat.

Manual has been updated and tests added.

Still left to do:

  • NetCDF4 files cannot be written in parallel yet; they require slightly different code than classic (which happens via pnetcdf).
  • The cmake build does not yet know how to link in HDF5.

Other changes in this PR:

  • Fix error detection in BufferedFrame::ReadFrame().
  • Fix error checking for Gzip IO (FileIO_Gzip).
  • Fix minor bug in the ArrayToRangeExpression() routine that could cause it to incorrectly fail (and also cause a minor memory issue).
  • New command compareenergy for comparing individual components of energy calculations. Still hidden, not ready for release yet.
  • Add Rattle() routine to Constraints class for potentially doing positions and velocities (not used yet).

Daniel R. Roe and others added 30 commits October 18, 2019 14:19
…tcdf symbols being pulled in from libsander.
…ing in unnecessary symbols (e.g. the netcdf symbols in the sander library).
@drroe drroe added enhancement bugfix New keywords New keywords for existing commands. labels Nov 16, 2022
@drroe drroe self-assigned this Nov 16, 2022
@AmberJenkins
Copy link
Collaborator

The PGI build in Jenkins failed.

@lgtm-com
Copy link

lgtm-com bot commented Nov 16, 2022

This pull request introduces 4 alerts when merging d69d7e9 into 81048b9 - view on LGTM.com

new alerts:

  • 2 for Wrong type of arguments to formatting function
  • 2 for Comparison result is always the same

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. Please enable GitHub code scanning, which uses the same CodeQL engine ⚙️ that powers LGTM.com. For more information, please check out our post on the GitHub blog.

@lgtm-com
Copy link

lgtm-com bot commented Nov 16, 2022

This pull request introduces 1 alert when merging 9e610ad into 81048b9 - view on LGTM.com

new alerts:

  • 1 for Comparison result is always the same

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. Please enable GitHub code scanning, which uses the same CodeQL engine ⚙️ that powers LGTM.com. For more information, please check out our post on the GitHub blog.

@lgtm-com
Copy link

lgtm-com bot commented Nov 16, 2022

This pull request introduces 1 alert when merging 662b48f into 81048b9 - view on LGTM.com

new alerts:

  • 1 for Comparison result is always the same

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. Please enable GitHub code scanning, which uses the same CodeQL engine ⚙️ that powers LGTM.com. For more information, please check out our post on the GitHub blog.

@lgtm-com
Copy link

lgtm-com bot commented Nov 17, 2022

This pull request introduces 1 alert when merging 5386f74 into 81048b9 - view on LGTM.com

new alerts:

  • 1 for Comparison result is always the same

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. Please enable GitHub code scanning, which uses the same CodeQL engine ⚙️ that powers LGTM.com. For more information, please check out our post on the GitHub blog.

@drroe
Copy link
Contributor Author

drroe commented Nov 17, 2022

Note: Appveyor failures were due to updates on the Appveyor platform itself. Reverting to the previous VS 2019 image fixed the build issues.

@drroe drroe merged commit 90cd8a4 into Amber-MD:master Nov 17, 2022
@drroe drroe deleted the netcdf4 branch November 17, 2022 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix enhancement New keywords New keywords for existing commands.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants