Releases: rappdw/fs-crawler
Releases · rappdw/fs-crawler
Add Reader that constructs graph using AbstractGraphBuilder
Implement Restart
Implement restart ability with Graph DB Implementation
DB Backed Graph Implementation
Move from in memory representation of graph to SQLite backed represen… …tation (#14) * define Graph as ABC and provide an in memory impl convert all code to use Graph ABC (except instantiation in cralwer) convert from using in memory sets to using generators * Create db backed implementation of graph using SQLite
Add keyring support
Password can now be retrieved via keyring. Service name is "fs-crawler"
Fix for pre 3.10 versions of python
v0.2.4 support pre 3.10 exception formatting
Validate graph before completing crawl
As the last step of a crawl, validate the graph and save .validated.x.csv files
Code Simplification, Prepare for checkpointing iterations
The code changes for this release are primarily for simplification and to prepare for implementing iteration checkpointing
Update to match FamilySearch API changes
The packages was updated to support FamilySearch API changes (resulted in HTTP 301 with old version).
Additionally, a few enhancements were made to reduce the number of calls made to the FamilySearch API
Restart/Continue Capable
This release has fixed a few bugs in the resolution of relationships. The major enhancement enables restart/continue from a previous run.
Preserve preferred name
Preserve preferred name for an individual Restructure crawler.py to allow for calling from another script