A re-implementation of redpony/cdec's tokenize-anything.pl script in python
samples/ is a bunch of data pulled from Wikipedia in a bunch of languages.
tok/ holds the same data, as tokenized by the original tokenize-anything.pl.
-
Notifications
You must be signed in to change notification settings - Fork 2
armatthews/TokenizeAnything
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
About
A re-implementation of redpony/cdec's tokenize-anything.pl script in python
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published