Migrate mwiki to Python3 #8
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mwiki python port wasn't behaving properly when run outside of the project root directory, and trying to find $cwd/$wiki which accidentally nested directories in the project.
Instead of going with executable scripts for template rendering we're using slightly simpler string formatting.
The DependencyGraph class handles the graph of dependencies. On edit, it automatically propagates change status to ensure fast queries when searching for the set of items that must be (re)built. Rather than always rebuilding the world, it provides convenience functions for rebuilding only a subset of the graph.
Adds a way to fetch the status of an entry. Converts mark_rebuilt checks into a database trigger.
Since the DependencyGraph code does a lot of conversion back and forth between rowid and name add a unique index to make those faster.
Adds a way to fetch all the items that have been marked changed without asking for the build ordering.
Adds simple key value storage interface.
We don't always need to load connected edges when removing one.
Usage of the Storage class isn't ever going to check for key existence, instead performing operations on the assumption that they will raise an exception if there's catastrophic error.
Adds the plugin interfaces that users will have to implement.
Adds DependencyGraph.has_dependency and DependencyGraph.dependencies to aid in working with dependencies.
Adds a light weight wrapper around Storage and DependencyGraph to handle the whole build management pipeline.
Storage uses NoDefault to detect unset default values such that None can be passed. Ensure that Storage shims have the same behaviour.
Mikins don't need to be named "PluginMixin"
Index functions for plugins renamed to clarify what they do, and added an aggregation function which pulls together the index. This trades space for time, and reduces the number of keys touched when extracting data from the index. This change also better mirrors the graph structure which goes: > entry -> <plugin> index <entry> -> <plugin> index -> <plugin> render <entry> -> render <entry> Previously, '<plugin> index' used to be considered "fake" and generating '<plugin> render <entry>' would load all of the '<plugin> index <entry>' keys, doing O(entries) reads on every plugin index update.
Were incorrectly using m.storage which is a function (prior the function and property collided in name).
The actual storage, and external storage function had a name collision. This resulted in callers being able to directly access storage and avoid updating the build graph or performing other operations they should not have.
part x / y, oops.
Rebuilding items actually causes the DependencyGraph to see them as being rebuilt.
Because Storage isn't guaranteed to have all the keys in the graph, we must iterate on the graph instead.
As was documented in the test: Storage.iterate only returns nodes that exist in storage, and nodes are _not_ guaranteed to be added to Storage unless the build has run. Whereas nodes are guaranteed to exist in DependencyGraph no matter what (otherwise they're not real). Moving the iteration ensures that even nodes without content are properly iterated over.
Catch cases where Plugin methods aren't implemented by raising exceptions.
Changed the graph structure for plugins with meta pages, to ensure
that they are correctly picked up and output into the index:
> ("plugin", name, "meta")
-> ("html", "~" + name)
-> ("file", "~" + name)
-> ("index")
Also fixes a few typo'd variables in the plugin building code.
Previously the page was built only using <br/> tags, now every <br/> tag includes a newline beside it. This makes it much more legible when reading in a text editor, and not through a browser.
Adds Mentions and Missing to the default template.
This removes a single pass over the input dictionaries.
Events index can be empty, handle that case.
Consumers should instead use the deps function provided to them, and filter by the prefix they want to match. This ensures that they get a consistent view of their dependencies, and can do something meaningful when their dependency has nothing in storage.
Remove plugins that have been completely ported to python. This breaks the non-python install, but this is expected.
It's convenient to dump the graph, added a command to do so.
Using json as a common format work well, but forcing plugins to split the content from each file just seems terrible. It adds an extra thing for a plugin implementer to remember.
"---" was forgotten, because the tests were never actually run...oops
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Convert the collection of Bash scripts, and Tup files into a collection of Python3 scripts.