v0.3.1

v0.3.1

Model

• Add drop path to regularize large models, and it works quite well for deep models
• Add EMA

Other

• Add one package dependency: timm, to implement EMA
• Update README to include details of Eulerian sequence and cyclic node re-index.
• Code refactoring.
• Tokenization config json refactoring.
• Update vocab by adding some special tokens, e.g., <bos>, <new>, <mask> and etc.
• Turn of optimizer offload in deepspeed config to boost the training speed.

v0.3.0

Full Changelog: v0.2.1...v0.3.0
see CHANGELOG.md for details.

v0.2.1

Full Changelog: v0.2.0...v0.2.1
fix bugs in issue: #2

implement permute nodes and refactor codes

v0.2.0

implement permute nodes and refactor codes

initial release with common-io bug fixed

v0.1.1

remove package common-io dependence because it is only used in Alibab…