No volatile #20
No volatile #20
Conversation
|
this needs bluesky/event-model#71 to pass |
suitcase/mongo_embedded/__init__.py
Outdated
| self._bulkwrite_datum({doc['resource']: doc}) | ||
| doc_size = len(bson.BSON.encode(doc)) | ||
| self._bulkwrite_datum({doc['resource']: doc}, | ||
| {doc['resource']: doc_size}) | ||
| return doc | ||
|
|
||
| def close(self): | ||
| self.freeze(self._run_uid) | ||
|
|
||
| def freeze(self, run_uid): |
There was a problem hiding this comment.
Should we retire the name "freeze" now? As your changes to the docstring suggestion, "finalize" might be a good name for what this now does. I have seen that term used in similar contexts.
There was a problem hiding this comment.
sounds good finalize is better
suitcase/mongo_embedded/__init__.py
Outdated
| @@ -186,7 +209,7 @@ def _event_worker(self): | |||
| do_push = False | |||
| try: | |||
| if event is None: | |||
| event = self._event_queue.get(timeout=0.5) | |||
| event = self._event_queue.get(timeout=0.2) | |||
There was a problem hiding this comment.
It's best to declare magic numbers like this as constants at the top of the module so that it's easy to quickly review the tunable parameters. See example: https://github.com/NSLS-II/caproto/blob/60327cda7d06203e1bf8db69d6e5014ae1276c2b/caproto/threading/client.py#L152-L160
There was a problem hiding this comment.
Looks good. Please do add a comment to this PR [I mean a comment here on GitHub, not a comment in the code] explaining the motivation for this change for others who may catch up on this later (current or future group members). I left some small suggest edits to update the docstring.
suitcase/mongo_embedded/__init__.py
Outdated
| @@ -49,17 +41,14 @@ class Serializer(event_model.DocumentRouter): | |||
| >>> from suitcase.mongo_embedded import Serializer | |||
|
|
|||
| >>> # Create two sandboxed mongo instances | |||
There was a problem hiding this comment.
| >>> # Create two sandboxed mongo instances | |
| >>> # Create a sandboxed Mongo instance. |
suitcase/mongo_embedded/__init__.py
Outdated
| >>> volatile_box.start() | ||
| >>> permanent_box.start() | ||
| >>> mongo_box = MongoBox() | ||
| >>> mongo_box.start() | ||
|
|
||
| >>> # Get references to the mongo databases |
There was a problem hiding this comment.
| >>> # Get references to the mongo databases | |
| >>> # Get a reference to the Mongo database. |
|
We decided that the volatile database was not necessary to ensure data integrity. We plan to use kafka, and with kafka, users won't have database write privileges. Instead they will publish their data to kafka. Without kafka this has similar data integrity to our current solution. We also are planning to archive each run from kafka in a TBD format in addition to writing to mongo. |
No description provided.