Major Utility Release

This release adds a number of new useful utilities to maggma

• Adds distributed processing for builders using pynng
• Adds a redesigned group_builder with documentation that should be easier to use
• Add build process reporting in mrun via a special LoggingHandler and extra info
• Updates the wording in most docs. Thanks @acrutt

Patch Release for Github Workflowws

This release is just to re-trggier the github workflow for releasing maggma
See v0.14.0 for any changes.

Major refactor of Maggma

This release refactors maggma to be better suited for regular releases.

Changes

• reorganized maggma modules into core, builders, and `stores.
  • Core defines the interfaces
  • the rest of modules provide implementations
• improved overall testing
  • added type hints and mypy type checking
  • added flake8 and pylint testing
  • added codecov checks
  • added LGTM checks
  • increased test coverage to 85%
• added base documentation
• rebuilt CLI using asyncio
• built CI workflows using github actions for testing, building documentation, and releases

Early November Release

• Moved and refactored builder to builders
• Moved MapBuilder to builders
• deprecated examples.builders
• Allow non-incremental building in MapBuilder
• Added updated_keys in Builder

We're very close to a v1.0 but there are still a few things to get right.

Various updates/improvements

• JointStore updates
• Validation improvements
• Added delete_orphans flag to MapBuilder to "sync" source and target stores, deleting target docs with keys no longer in source store docs.
• Fix: SerialProcessor.process now calls builder.finalize

Thanks @mkhorton @shyamd

Added a new JointStore

• A new JointStore was added. This Store will aggregate across stores to create a single interface for querying.
• Fixed issues with tqdm and interactive shells
• switched the order of properties and criteria in query to match that of regular pymongo find calls
• fix/simplify the JSON validator

Improved Incremental Building; Added MapBuilder example

• Added MapBuilder -- simply write a function to map a source store document to a target store document. Supports automatic incremental building at the document level that is resilient to builder interruptions. Intended usage: subclass MapBuilder.
• Added GroupBuilder as an extension of MapBuilder -- group source docs and produce one target doc from each group.
• breaking change: Runner parameter num_workers renamed to max_workers to better reflect function.

Added MongograntStore

Using mongogrant allows serialization of builders/runners without username/password credentials.

Refined GridFS store

We've got a refined GridFS store from the Alsace region. It's of mint variety and properly stores keys in the metadata and handles gzip compression internally.

• GridFS Store is now compliant with mongo specs
• GridFS store also handles compression internally
• Added in progress bars
• Fixed a potential race condition in the multiprocessing

New S3 Store and some fixes

We're starting to expand functionality. One of the keys for Maggma will be connecting to a lot of different data sources but leveraging the powerfull querying in MongoDB. The GridFS concept of an index + bucket storage patterns really well to other cloud storage platforms that we hope to support.

Changes:

• Added an AmazonS3 Store
• Added from_collection in mongostore to instantiate a MongoStore from a PyMongo Collection object
  Warning: This object will not serialize and deserialize properly so don't expect this to work with MPI building or saving to files
• Fixed distinct and groupby to be applicable to all stores.