Skip to content

Commit

Permalink
Bump version
Browse files Browse the repository at this point in the history
  • Loading branch information
Shyam Dwaraknath committed Nov 19, 2018
1 parent 1f875fc commit 32da425
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 1 deletion.
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
if __name__ == "__main__":
setup(
name="maggma",
version="0.11.0",
version="0.12.0",
description="MongoDB aggregation machine",
long_description=open(os.path.join(module_dir, "README.md")).read(),
url="https://github.com/materialsproject/maggma",
Expand Down
11 changes: 11 additions & 0 deletions source/usage/introduction.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
=================
Introduction
=================

.. highlight:: console

Maggma is designed to provide a consistent interface to a variety of data sources using a standard Mongo-style collections and methods. Many of the internal of Maggma leverage pyMongo explicitly or pass out cursors and other pyMongo objects. Several advanced Maggma interfaces simplify the the process of connecting to data sources such as Mongo's internal GridFS or AWS block storage. A mongo database is still required. Other interfaces provide implicit aggregation, sandboxing, or aliasing to enable complex data access patterns.

An important aspect of Maggma is to provide a framework to describe data manipulation, particuarly in the context of complex data pipelines such as those employed in Material Science. Maggma enables these pipelines to scale by providing a "Builder" interface that can be used to construct a data operation between Maggma Stores. The "Runner" class in Maggma enables deploying these "Builder"s to various resources whether it is simple serial processing or multiprocessing across cores on a single machine or multiprocessing across computers using MPI.

Finally, everything in Maggma is designed to serializable to json. The monty MSONable pattern is employed so that stores, builders, and runners can be easily coverted to python dictionaries and back or even dumped quickly to file and loaded back. This ensures that pipelines can be easily saved and rerun at later times, ensuring explicit documentation of a data manipulation process.

0 comments on commit 32da425

Please sign in to comment.