Mixin Design Pattern approach

[egberts]

I attempted to explore and implement a Mixin Design Pattern as suggested in #37 . In the long continuum of trying to incorporate additional support into Pyparsing the following:

• output a file of the near-original input file from its Pyparsing-constructed dict/list results
• provide a .get/.set its value settings in each of its own Pyparsing-specific Python class

I took recursion statement, a simple boolean setting found in options clause of named.conf configuration file for ISC Bind9.

A working snippet of Mixin-class for recursion (boolean) setting.

import pyparsing
from pyparsing import *


class WrapperPyparsing(pyparsing_common):

    # _parent_parser_element_built  # Parser constructors
    # _parent_parser_element_after  # Parser constructors
    # _parent_parser_element_before  # Parser constructors
    # _parser_element: None  # Parser constructors
    _keyword_name_str: str

    def __init__(self):
        self._keyword_name_str: str = ''
        self._parser_element: str = ''  # Parse constructors
        self._parent_parser_element_before: str = ''  # Parse constructors
        self._parent_parser_element_after: str = ''  # Parse constructors
        self._parent_parser_element_built: str = ''  # Parse constructors

    def add_parse_element(
            self,
            arg_keyword_name_str: str,
            arg_parser_element: str,
            arg_parent_parser_element_before: str = None,
            arg_parent_parser_element_after: str = None):
        """
        Construct the Pyparsing token tree with EBNF details of its configuration setting
        :type arg_keyword_name_str: str
        :param: arg_parse_element - A `ParseElement` class containing details about a specific EBNF
        :type arg_parser_element: ParserElement
        :param arg_parser_element: The name of the keyword associated with this ParseElement
        :type arg_parent_parser_element_before: ParserElement
        :param arg_parent_parser_element_before:  - A `ParseElement` who is a parent for this `ParseElement`
        :type arg_parent_parser_element_after: ParserElement
        :param arg_parent_parser_element_after:  - A `ParseElement` who is a parent for this `ParseElement`
        """
        self._keyword_name_str = arg_keyword_name_str
        self._parser_element = arg_parser_element
        self._parent_parser_element_before = arg_parent_parser_element_before
        self._parent_parser_element_after = arg_parent_parser_element_after
        if arg_parent_parser_element_after is None:
            if arg_parent_parser_element_before is None:
                self._parent_parser_element_built = arg_parser_element
            else:
                self._parent_parser_element_built = arg_parent_parser_element_before + arg_parser_element
        else:
            if arg_parent_parser_element_before is None:
                self._parent_parser_element_built = arg_parser_element + arg_parent_parser_element_after
            else:
                self._parent_parser_element_built = \
                        arg_parent_parser_element_before\
                        + arg_parser_element\
                        + arg_parent_parser_element_after
        self._parent_parser_element_built.setName(arg_keyword_name_str)

    def get_parse_elements(self):
        return self._parser_element

    def get_parent_parse_elements(self):
        return self._parent_parser_element_built

    def load_parse_elements(self, arg_parse_elements: dict):
        """
        :param: arg_parse_elements - A dict type variable containing tokenized content of 
        named.conf configuration settings
        """
        pass

    def load_text_config(self, arg_text_named_conf: str):
        """
        A shortcut to load a snippet of named.conf configuration file, but it requires a prior knowledge 
        of what keyword it contains before calling this method using the correct EBNF declarator class.
        Useful for unit testing and those who possess a config file that
        uses `include` pragma statement(s).
        :param: arg_text_named_conf - A text-based ISC Bind9 configuration string or portion thereof
        """
        pass

    def output_text_config(self, filename: str, arg_parse_elements: dict):
        """
        Basically formats the output string of the final `named.conf` configuration  settings based
        on the current holding of class-specific token variable that it currently holds within its class.
        :param: filespec: str - a string containing the file path specification in which to write 
            the named.conf configuration settings into
        :param: arg_parse_elements: dict - a dict type variable containing tokenized 
            content of named.conf configuration settings
        """
        pass


class ParentWrapperPyparsing(WrapperPyparsing):

    master_parser_elements: ParserElement = {}

    def __init__(self):
        """
        Initialize and builds all the `ParseElement` classes together.
        """
        super().__init__()
        self._desired_config_version: str = ''

    def desired_config_version(self, arg_desired_config_version: str):
        self._desired_config_version = arg_desired_config_version
        
    def encountered_config_version(self) -> str: pass
    def read_config_file(self, arg_filespec: str):  pass
    def get_token_elements(self) -> dict: pass
    def set_token_elements(self, arg_token_elements: dict): pass
    def output_config_file(self, arg_filespec: str): pass


ncpe = ParentWrapperPyparsing()
ncpe.desired_config_version('9.19.1')

options_stmt_recursion = (
        Keyword('recursion').suppress()
        + (
                Word('yes')
                ^ Word('no')
        )('recursion')
        + Char(';').suppress()
).setName('recursion')

pp_options = WrapperPyparsing()
pp_options.add_parse_element('recursion', arg_parser_element=options_stmt_recursion)

options_statements_set = (
    pp_options.get_parse_elements()
    # ^ options_stmt_acache_enable
)

options_all_statements_set = (
        options_statements_set
        # ^ optview_statements_set
)

options_all_statements_series = (
    ZeroOrMore(
        options_all_statements_set
    )
)

clause_stmt_options_standalone_pre = (
    Keyword('options').suppress()
    - Char('{').suppress()
    - Group(
        options_all_statements_series
    )('options*')
    + Char('}').suppress()
    + Char(';').suppress()
)


ncpe.add_parse_element(
    arg_parser_element=clause_stmt_options_standalone_pre,
    arg_keyword_name_str='options')

pe_options = ncpe.get_parent_parse_elements()
my_ncpe = pe_options()

named_conf_content = 'options { recursion yes; };'
# named_conf_content = 'options { };'
parse_result_list = my_ncpe.parseString(named_conf_content, parseAll=False)
print('list: %s' % parse_result_list)


parse_result_dict = parse_result_list.asDict()
import pprint
pp = pprint.PrettyPrinter(width=2, compact=False, indent=1)
print('\nprint(result.asDict()):')
pp.pprint(parse_result_list.asDict())

print('done')

resulted in demonstrating the holding of Pyparsing (ParserElement) string:

list:  [['yes']]

print(result.asDict()):
{'options': [{'recursion': 'yes'}]}
done

The only problem I have is how to store that boolean result back into the class so that various user interface (UI) and graphical unser interface (UX) can then make suitable retrieval and settings by end-users. @ptmcg

Where does one extract the info so that these .get and .set methods can be supported? set_parse_action ?

I've got some 2,400 str elements to retrofit so prototyping is paramount before deployment.

[egberts]

Perhaps something like a highly-customized .dump() for each "class"? Where one can

• output the settings, in the usual single-line, (auto-indented), terminated-with-semicolon-symbol manner.
• Group() may entail the use of outputting curly braces around a set.
• write to a string or file

Of course, to do all that, I need to start deploying one Python class for each aspect of the EBNF and use some form of copy.deepcopy() to ensure no overloading or blowning away of nested settings.

Seems like my biggest hangup is the following:

• Class-ify a standard module? Or break them into types of named.conf settings (ie. isc_boolean)
• Add a .dump() for each types of named.conf
• Provide a .get/.set for a nicer UI/UX.

I've trashed some 8-odd prototypings on Mixin and still pouring over the READTHEDOCS.

[ptmcg]

I finally got to put some time into this late last evening, to do some experimenting with a mixin class, to support enhancing the imported pyparsing classes using a pretty intrusive monkeypatch. I didn't really follow what it was you wanted to add to the pyparsing classes themselves, so I added parent, next, and previous pointers as an experiment. [EDIT - the pointers are not included in this example, instead I just added a function to the base ParserElement class.]

In doing so, I found that I was doing another bad practice in the Word/_WordRegex classes, assigning to __class__. I just committed and pushed a change to remove that (and was able to completely remove _WordRegex and also simplify the code in Char).

So here is my "working" mixin example:

import pyparsing as pp


class EnhancedParserElement(pp.ParserElement):

    def added_function(self):
        return repr(f"{self} - {hex(id(self))}")


orig_parser_element = pp.ParserElement
pp.ParserElement = EnhancedParserElement


def type_with_modified_mro(cls):
    cur_mro = cls.__mro__
    # don't patch if already patched
    if EnhancedParserElement in cur_mro:
        return cls

    # compose new MRO replacing ParserElement with EnhancedParserElement
    new_mro = tuple(EnhancedParserElement if klass is orig_parser_element else klass for klass in cur_mro)

    return type(cls.__name__, new_mro, {})


def monkeypatch_all_parserelement_classes():
    from collections import deque

    # use deque instead of recursion to process inheritance tree
    to_patch = deque(orig_parser_element.__subclasses__())
    while to_patch:
        klass = to_patch.popleft()
        # print(klass.__name__)

        # monkeypatch pyparsing module, replacing types with modified types
        setattr(pp, klass.__name__, type_with_modified_mro(klass))

        to_patch.extend(klass.__subclasses__())


# print(pp.Word.__mro__)
monkeypatch_all_parserelement_classes()
# print(pp.Word.__mro__)


# try it out!
integer = pp.Word(pp.nums)
string = pp.Word(pp.alphas, pp.alphas + ".")
# this won't work, must use OneOrMore explicitly
# street_name = string[1, ...]
street_name = pp.OneOrMore(string)
street_address = integer + street_name

for e in street_address.exprs:
    print(e.added_function())

To do more work like adding a pointer to the parent element, things get a little trickier, but still mostly manageable. The point of the mixin is to minimize the impact to your actual grammar, and hack the underlying pyparsing classes.

Is this something that might work for you?

(Let me know when you get to adding the parent pointer - I'm still working on cleaning that up.)

-- Paul

P.S. - Why are you subclassing from pyparsing_common? This class has no behavior, it is just a namespace container for common pyparsing functions (same story for pyparsing_unicode). Namespace classes are not really Pythonic, but they are what I came up with to keep the overall pyparsing namespace down in size.

[egberts]

This is a good starting point for me to work on next.

Minor sidenote: something else that came to mind that was frequently occurring but only remains a minor issue (only because once understood, it is now attainable and available to all):

I’ve made frequent use of pp_str(‘variable_keyword_name*’) often paired with Group() and sometimes without the asterisk symbol when attempting to aggregate into an indexable rows of named dict/list. Perhaps a new Python method for both because this one tiny but relatively obscure mechanism is what makes the whole thing possible (JSON-able, dict/list).

As to my mysterious subclassing attempt to include pyparsing_common, I am but merely an intermediate Python user still learning how to incorporate each known OOP design methods … in Python in effort to realize its tangible benefit (without so much focus on its uselessness of such effort). This step was an attempt by me to see if anything tangible can be had by incorporating new types of parsing (such as isc_boolean that supports not only true/false, but also yes/no, and 1/0.). Also an attempt to see if I could modularize some things that can be reused elsewhere (such as the complex addresses_match_list (AML) that is recursive. I now realize that I would be dragging in too much of the pyparsing along with its whole kaboodle.

given that I actually have a 100% working Pyparsing of a rather large configuration file, in retrospect, I noticed a lot of pp_str construction and starting to see a pattern emerging:

• when attempting to write them back out into a file (in its original file format), there is a common theme (curly braces output always have a Group() method but Group() may not always output curly braces so I know that I could either supplant with a new GroupCurlyBraces() method or subclass this.
• some pp_str ends with semicolon; I could subclass that but .print/.dump has to be introduced in this new class.
• a need to hold a EBNF set (in form of pp_str) within a Python class (not only for readability but reusability and maintainability) but I have yet to figure out a game plan on when to construct the entire super pp_str from all classes before the famed parseString(). That’s but an entirely separate design issue not on the forefront here because it entails expected version of EBNF, all-version and auto-detected versioning.
• a need to construct a Python class to hold this output/dump at its most finest granularity level (preferably at each config option)
• A need to provide end-user with minor edit of Python dict/list
• A need to hold such setting values if there is a class for each setting for uses by ordinary Python programming of “business logic”

of course, I could always do the shortcut and just write a nice config outputter in a stand-alone fashion but setting/getting is better served by this further classing/subclassing/sub-subclassing (or better, pp_str.method().method().method()) and would still not be satisfied by this straightforward but long outputter (spaghetti) code but I am not ready to give this up yet because I see “pattern attainablity” in working this not only .get()/.set() but a suitable .print()/.dump that would actually sell Pyparsing even further.

So in essence, enters my not-so-practical attempt to subclass pyparsing_common with a goal to envelope the above new functionalities reiterated here and listed in data flow ordering:

.load_pp_str()
.load_data_content()
.intrinsic_parse_string()
.get()
.set()
.print()/.dump() # outputter

[egberts]

I created a design doc in #41 in attempt to narrow down the goal of creating outputters/dumpers. some insight might be helpful there.