diff --git a/docs/api_reference/trestle.core.jinja.base.md b/docs/api_reference/trestle.core.jinja.base.md new file mode 100644 index 000000000..b929616d6 --- /dev/null +++ b/docs/api_reference/trestle.core.jinja.base.md @@ -0,0 +1,2 @@ +::: trestle.core.jinja.base +handler: python diff --git a/docs/api_reference/trestle.core.jinja.ext.md b/docs/api_reference/trestle.core.jinja.ext.md new file mode 100644 index 000000000..5da0b570e --- /dev/null +++ b/docs/api_reference/trestle.core.jinja.ext.md @@ -0,0 +1,2 @@ +::: trestle.core.jinja.ext +handler: python diff --git a/docs/api_reference/trestle.core.jinja.filters.md b/docs/api_reference/trestle.core.jinja.filters.md new file mode 100644 index 000000000..40a4d4e5e --- /dev/null +++ b/docs/api_reference/trestle.core.jinja.filters.md @@ -0,0 +1,2 @@ +::: trestle.core.jinja.filters +handler: python diff --git a/docs/api_reference/trestle.core.jinja.md b/docs/api_reference/trestle.core.jinja.md deleted file mode 100644 index ea057f7e1..000000000 --- a/docs/api_reference/trestle.core.jinja.md +++ /dev/null @@ -1,2 +0,0 @@ -::: trestle.core.jinja -handler: python diff --git a/docs/api_reference/trestle.core.jinja.tags.md b/docs/api_reference/trestle.core.jinja.tags.md new file mode 100644 index 000000000..53e6638a0 --- /dev/null +++ b/docs/api_reference/trestle.core.jinja.tags.md @@ -0,0 +1,2 @@ +::: trestle.core.jinja.tags +handler: python diff --git a/docs/api_reference/trestle.core.plugins.md b/docs/api_reference/trestle.core.plugins.md new file mode 100644 index 000000000..54e01680b --- /dev/null +++ b/docs/api_reference/trestle.core.plugins.md @@ -0,0 +1,2 @@ +::: trestle.core.plugins +handler: python diff --git a/docs/contributing/plugins.md b/docs/contributing/plugins.md index f4f7d6582..89330caf6 100644 --- a/docs/contributing/plugins.md +++ b/docs/contributing/plugins.md @@ -13,17 +13,22 @@ The plugin project should be organized as shown below. ```text compliance-trestle-fedramp ├── trestle_fedramp -│ ├── __init.py__ +│ ├── __init__.py │ ├── commands -| | ├── __init.py__ +| | ├── __init__.py | | ├── validate.py +| ├── jinja_ext +| | ├── __init__.py +| | ├── filters.py │ ├── ├── ``` Trestle uses a naming convention to discover the top-level module of the plugin projects. It expects the top-level module to be named `trestle_{plugin_name}`. This covention must be followed by plugins to be discoverable by trestle. In the above example, the top-level module is named as `trestle_fedramp` so that it can be autmatically discovered by trestle. All the python source files should be created inside this module (folder). -The top-evel module should contain a `commands` directory where all the plugin command files should be stored. Each command should have its own python file. In the above exaample, `validate.py` file conatins one command for this plugin. Other python files or folders should be created in the top-level module folder, outside the `commands` folder. This helps in keeping the commands separate and in their discovery by trestle. +To add commands to the CLI interface, the top-level module should contain a `commands` directory where all the plugin command files should be stored. Each command should have its own python file. In the above example, `validate.py` file contains one command for this plugin. Other python files or folders should be created in the top-level module folder, outside the `commands` folder. This helps in keeping the commands separate and in their discovery by trestle. + +To add jinja extensions available during `trestle author jinja`, the top-level module should contain a `jinja_ext` directory where all extension files should be stored. Each extension should have its own python file. In the above example, `filters.py` file contains a single extension class, which may define many filters or custom tags. Supporting code should be created in the top-level module folder, outside the `jinja_ext` folder. This helps in keeping the extensions separate and in their discovery by trestle. ## Command Creation @@ -61,3 +66,28 @@ There should be a command class for example, `ValidateCmd` which should either e The docstring of the command class is used as the help message for the command. Input arguments to the command should be specified in `_init_arguments` method as shown above. The acutal code of the command is contained in`_run` method. This method is called by ilcli when the command is excuted on the commandline. The command arguments can be accessed from the `args` input parameter as shown above. The command should return `0` in case of successful execution, or any number greater than 0 in case of failure. Please see `trestle.core.commands.common.return_codes.CmdReturnCodes` class for specific return codes in case of failure. The command class should conatin the `name` field which should be set to the desired command name. In the above example, the command is called `fedramp-validate`. This name is automatically added to the list of sub-command names of trestle during the plugin discovery process. This command can then be invoked as `trestle {name}` from the commandline e.g., `trestle fedramp-validate`. Any input parameters to the command can also be passed on the commandline after the command name. + +## Jinja Extension Creation + +The plugin extension should be created as shown in the below code snippet. + +```python +from jinja2 import Environment +from trestle.core.jinja.base import TrestleJinjaExtension + +def _mark_tktk(value: str) -> str: + """Mark a value with TKTK to easily find it for future revision.""" + return f'TKTK {value} TKTK' + + +class Filters(TrestleJinjaExtension): + def __init__(self, environment: Environment) -> None: + super(Filters, self).__init__(environment) + + environment.filters['tktk'] = _mark_tktk + +``` + +There should be an extension class, for example `Filters` that must extend from `TrestleJinjaExtension` or `jinja2.ext.Extention`. The `__init__` method must call init for its superclass. Beyond that, any behavior for standard [jinja2 custom extensions](https://jinja.palletsprojects.com/en/3.1.x/extensions/#module-jinja2.ext) is supported. + +Examples for implementing extensions can be found at `trestle/core/jinja/tags.py` and `trestle/core/jinja/filters.py` diff --git a/docs/trestle_author_jinja.md b/docs/trestle_author_jinja.md index 45001f513..9f9df1349 100644 --- a/docs/trestle_author_jinja.md +++ b/docs/trestle_author_jinja.md @@ -113,7 +113,7 @@ Trestle provides custom jinja tags for use specifically with markdown: `mdsectio 1. `{% md_clean_include 'path_to_file.md' heading_level=2 %}` 1. The heading level argument adjusts to (based on the number of hashes) the most significant heading in the document, if headings exist. -`mdsection_include` is similar to the native `md_clean_include` except that.: +`mdsection_include` is similar to `md_clean_include` except that: 1. `mdsection_include` requires an second positional argument which is the title of a heading, from a markdown file, which you want the content from. @@ -129,6 +129,23 @@ Trestle provides custom jinja tags for use specifically with markdown: `mdsectio 1. `format` where a python [datetime strftime format string](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) is provided to format the output. E.g. `{% md_datestamp format='%B %d, %Y' %}` results in `December 28, 2021` being inserted. 1. `newline` is a boolean to control the addition of a double newline after the inserted date string. For example `{% md_datestamp newline=false %}` inserts a date in the default format, without additional newlines. +## Custom Jinja filters. + +Trestle provides custom jinja filters for help processing SSP data. + +- `as_list` will return the passed value, or an empty list if `None` is passed in. + - Example: `{% for party in ssp.metadata.parties | as_list %}` +- `get_default` operates the same as the built-in Jinja `default` filter, with the optional second parameter set to `True` + - Example: `{{ control_interface.get_prop(user, 'user-property') | get_default('[Property]') }}` +- `first_or_none` will return the first element of a list, or `None` if the list is empty or `None` itself. + - Example: `{% set address = party.addresses | first_or_none %}` +- `get_party` will return the `Party` found in `ssp.metadata.parties` for a given `uuid` + - Example: `{% set organization = party.member_of_organizations | first_or_none | get_party(ssp) %}` +- `parties_for_role` will yield individual `Party` entries when given a list of `ResponsibleParty` and a `role-id` + - Example: `{% for party in ssp.metadata.responsible_parties | parties_for_role("prepared-by", ssp) %}` +- `diagram_href` will return the `Link.href` where `Link.rel == 'diagram'` when given a `Diagram` object + - Example: `![{{diagram.caption}}]({{ diagram | diagram_href }})` + ## Generate controls as individual markdown pages. Trestle's Jinja functionality allows its users to generate individual markdown pages for each control from a resolved profile catalog. Such functionality can be used later on to pack individual pages into docs of various formats. @@ -144,17 +161,17 @@ To achieve that, we can create a simple Jinja template that would be used to gen {{ control_writer.write_control_with_sections( control, profile, - group_title, - ['statement', 'objective', 'expected_evidence', 'implementation_guidance', 'table_of_parameters'], + group_title, + ['statement', 'objective', 'expected_evidence', 'implementation_guidance', 'table_of_parameters'], { 'statement':'Control Statement', - 'objective':'Control Objective', - 'expected_evidence':'Expected Evidence', - 'implementation_guidance':'Implementation Guidance', + 'objective':'Control Objective', + 'expected_evidence':'Expected Evidence', + 'implementation_guidance':'Implementation Guidance', 'table_of_parameters':'Control Parameters' } - ) - + ) + | safe }} ``` diff --git a/mkdocs.yml b/mkdocs.yml index 22d3dd98b..0cbad372e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -132,7 +132,11 @@ nav: - duplicates_validator: api_reference/trestle.core.duplicates_validator.md - generators: api_reference/trestle.core.generators.md - generic_oscal: api_reference/trestle.core.generic_oscal.md - - jinja: api_reference/trestle.core.jinja.md + - jinja: + - base: api_reference/trestle.core.jinja.base.md + - ext: api_reference/trestle.core.jinja.ext.md + - filters: api_reference/trestle.core.jinja.filters.md + - tags: api_reference/trestle.core.jinja.tags.md - links_validator: api_reference/trestle.core.links_validator.md - markdown: - base_markdown_node: api_reference/trestle.core.markdown.base_markdown_node.md @@ -152,6 +156,7 @@ nav: - object_factory: api_reference/trestle.core.object_factory.md - parser: api_reference/trestle.core.parser.md - pipeline: api_reference/trestle.core.pipeline.md + - plugins: api_reference/trestle.core.plugins.md - profile_resolver: api_reference/trestle.core.profile_resolver.md - refs_validator: api_reference/trestle.core.refs_validator.md - remote: diff --git a/tests/conftest.py b/tests/conftest.py index d4053b29d..21704a0f8 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -32,8 +32,10 @@ from trestle.cli import Trestle from trestle.common.err import TrestleError from trestle.oscal import catalog as cat +from trestle.oscal.common import Party from trestle.oscal.component import ComponentDefinition, DefinedComponent from trestle.oscal.profile import Profile +from trestle.oscal.ssp import SystemSecurityPlan TEST_CONFIG: dict = {} @@ -190,6 +192,20 @@ def sample_catalog_subgroups(): return catalog_obj +@pytest.fixture(scope='function') +def sample_party() -> Party: + """Return a valid Party object.""" + return gens.generate_sample_model(Party, True, 3) + + +@pytest.fixture(scope='function') +def sample_system_security_plan(sample_party: Party) -> SystemSecurityPlan: + """Return a valid SSP object with some contents.""" + ssp: SystemSecurityPlan = gens.generate_sample_model(SystemSecurityPlan, True, 2) + ssp.metadata.parties = [gens.generate_sample_model(Party, True, 3), sample_party] + return ssp + + @pytest.fixture(scope='function') def sample_component_definition(): """Return a valid ComponentDefinition object with some contents.""" diff --git a/tests/trestle/core/jinja/filters_test.py b/tests/trestle/core/jinja/filters_test.py new file mode 100644 index 000000000..24848bf24 --- /dev/null +++ b/tests/trestle/core/jinja/filters_test.py @@ -0,0 +1,69 @@ +# -*- mode:python; coding:utf-8 -*- + +# Copyright (c) 2024 The OSCAL Compass Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Tests for trestle custom jinja filters functionality.""" + +from typing import Any, List, Optional + +import pytest + +from trestle.core.jinja.filters import (diagram_href, first_or_none, get_party, parties_for_role) +from trestle.oscal.common import Link, Party, ResponsibleParty +from trestle.oscal.ssp import Diagram, SystemSecurityPlan + + +@pytest.mark.parametrize( + 'links,expected', + [ + ( + [ + Link(rel='other', href='./path/to/local/thing'), + Link(rel='diagram', href='https://host.name/path/to/diagram.png') + ], + 'https://host.name/path/to/diagram.png' + ), ([Link(rel='other', href='./path/to/local/file')], ''), ([], ''), (None, '') + ] +) +def test_diagram_href(links: Optional[List[Link]], expected: str) -> None: + """Test retrieving the link href for rel='diagram'.""" + diagram = Diagram(uuid='26c1c7df-fb67-45ba-b60f-35d8b5c1d1dc', links=links) + assert diagram_href(diagram) == expected + + +@pytest.mark.parametrize('actual,expected', [[['ok'], 'ok'], ([], None), (None, None)]) +def test_first_or_none(actual: Optional[List[Any]], expected: Optional[Any]) -> None: + """Test behavior of retrieving the first element or None for empty or missing list.""" + assert first_or_none(actual) == expected + + +def test_get_party(sample_system_security_plan: SystemSecurityPlan, sample_party: Party) -> None: + """Test behavior of retrieving a ssp.metadata.parties entry by UUID.""" + assert get_party(sample_party.uuid, ssp=sample_system_security_plan) == sample_party + + +def test_parties_for_role(sample_system_security_plan: SystemSecurityPlan, sample_party: Party) -> None: + """Test behavior of retrieving all parties for a given role-id.""" + sample_system_security_plan.metadata.responsible_parties = [ + ResponsibleParty(role_id='pytest-tester', party_uuids=[sample_party.uuid]) + ] + result = list( + parties_for_role( + sample_system_security_plan.metadata.responsible_parties, + role_id='pytest-tester', + ssp=sample_system_security_plan + ) + ) + assert len(result) == 1 + assert result[0] == sample_party diff --git a/tests/trestle/core/jinja_test.py b/tests/trestle/core/jinja/tags_test.py similarity index 98% rename from tests/trestle/core/jinja_test.py rename to tests/trestle/core/jinja/tags_test.py index 09a888ed4..0885ad047 100644 --- a/tests/trestle/core/jinja_test.py +++ b/tests/trestle/core/jinja/tags_test.py @@ -22,7 +22,7 @@ import pytest -import trestle.core.jinja as tres_jinja +import trestle.core.jinja.tags as tres_jinja from trestle.core.markdown import markdown_const JINJA_MD = 'jinja_markdown_include' diff --git a/trestle/cli.py b/trestle/cli.py index badb563f1..918b0e711 100644 --- a/trestle/cli.py +++ b/trestle/cli.py @@ -14,11 +14,8 @@ # limitations under the License. """Starting point for the Trestle CLI.""" -import importlib -import inspect import logging import pathlib -import pkgutil from trestle.common import const, log from trestle.core.commands.assemble import AssembleCmd @@ -38,6 +35,7 @@ from trestle.core.commands.task import TaskCmd from trestle.core.commands.validate import ValidateCmd from trestle.core.commands.version import VersionCmd +from trestle.core.plugins import discovered_plugins logger = logging.getLogger('trestle') @@ -63,29 +61,14 @@ class Trestle(CommandBase): VersionCmd ] - discovered_plugins = { - name: importlib.import_module(name) - for finder, - name, - ispkg in pkgutil.iter_modules() - if name.startswith('trestle_') - } - - logger.debug(discovered_plugins) # This block is uncovered as trestle cannot find plugins in it's unit tests - it is the base module. - for plugin, value in discovered_plugins.items(): # pragma: nocover - for _, module, _ in pkgutil.iter_modules([pathlib.Path(value.__path__[0], 'commands')]): - logger.debug(module) - command_module = importlib.import_module(f'{plugin}.commands.{module}') - clsmembers = inspect.getmembers(command_module, inspect.isclass) - logger.debug(clsmembers) - for _, cmd_cls in clsmembers: - # add commands (derived from CommandPlusDocs or CommandBase) to subcommands list - if issubclass(cmd_cls, CommandBase): - # don't add CommandPlusDocs or CommandBase - if cmd_cls is not CommandPlusDocs and cmd_cls is not CommandBase: - subcommands.append(cmd_cls) - logger.info(f'{cmd_cls} added to subcommands from plugin {plugin}') + for plugin, cmd_cls in discovered_plugins('commands'): # pragma: nocover + # add commands (derived from CommandPlusDocs or CommandBase) to subcommands list + if issubclass(cmd_cls, CommandBase): + # don't add CommandPlusDocs or CommandBase + if cmd_cls is not CommandPlusDocs and cmd_cls is not CommandBase: + subcommands.append(cmd_cls) + logger.info(f'{cmd_cls} added to subcommands from plugin {plugin}') def _init_arguments(self) -> None: self.add_argument('-v', '--verbose', help=const.DISPLAY_VERBOSE_OUTPUT, action='count', default=0) diff --git a/trestle/core/commands/author/jinja.py b/trestle/core/commands/author/jinja.py index 56d01ca5e..55b2752a8 100644 --- a/trestle/core/commands/author/jinja.py +++ b/trestle/core/commands/author/jinja.py @@ -35,7 +35,7 @@ from trestle.core.commands.common.return_codes import CmdReturnCodes from trestle.core.control_interface import ControlInterface, ParameterRep from trestle.core.docs_control_writer import DocsControlWriter -from trestle.core.jinja import MDCleanInclude, MDDatestamp, MDSectionInclude +from trestle.core.jinja.ext import extensions from trestle.core.profile_resolver import ProfileResolver from trestle.core.ssp_io import SSPMarkdownWriter from trestle.oscal.profile import Profile @@ -191,10 +191,7 @@ def jinja_ify( """Run jinja over an input file with additional booleans.""" template_folder = pathlib.Path.cwd() jinja_env = Environment( - loader=FileSystemLoader(template_folder), - extensions=[MDSectionInclude, MDCleanInclude, MDDatestamp], - trim_blocks=True, - autoescape=True + loader=FileSystemLoader(template_folder), extensions=extensions(), trim_blocks=True, autoescape=True ) template = jinja_env.get_template(str(r_input_file)) # create boolean dict @@ -283,7 +280,7 @@ def jinja_multiple_md( jinja_env = Environment( loader=FileSystemLoader(template_folder), - extensions=[MDSectionInclude, MDCleanInclude, MDDatestamp], + extensions=extensions(), trim_blocks=True, autoescape=True ) @@ -315,7 +312,7 @@ def render_template(template: Template, lut: Dict[str, Any], template_folder: pa dict_loader = DictLoader({str(random_name): new_output}) jinja_env = Environment( loader=ChoiceLoader([dict_loader, FileSystemLoader(template_folder)]), - extensions=[MDCleanInclude, MDSectionInclude, MDDatestamp], + extensions=extensions(), autoescape=True, trim_blocks=True ) diff --git a/trestle/core/jinja/__init__.py b/trestle/core/jinja/__init__.py new file mode 100644 index 000000000..348baa8dc --- /dev/null +++ b/trestle/core/jinja/__init__.py @@ -0,0 +1,16 @@ +# -*- mode:python; coding:utf-8 -*- + +# Copyright (c) 2024 The OSCAL Compass Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Trestle core.jinja functionality.""" diff --git a/trestle/core/jinja/base.py b/trestle/core/jinja/base.py new file mode 100644 index 000000000..5e851f149 --- /dev/null +++ b/trestle/core/jinja/base.py @@ -0,0 +1,48 @@ +# -*- mode:python; coding:utf-8 -*- + +# Copyright (c) 2024 The OSCAL Compass Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Trestle core.jinja base class.""" +from jinja2 import lexer, nodes +from jinja2.environment import Environment +from jinja2.ext import Extension + + +class TrestleJinjaExtension(Extension): + """Class to define common methods to be inherited from for use in trestle.""" + + max_tag_parse = 20 + + def __init__(self, environment: Environment) -> None: + """Ensure enviroment is set and carried into class vars.""" + super().__init__(environment) + + @staticmethod + def parse_expression(parser): + """Safely parse jinja expression.""" + # Licensed under MIT from: + # https://github.com/MoritzS/jinja2-django-tags/blob/master/jdj_tags/extensions.py#L424 + # Due to how the jinja2 parser works, it treats "foo" "bar" as a single + # string literal as it is the case in python. + # But the url tag in django supports multiple string arguments, e.g. + # "{% url 'my_view' 'arg1' 'arg2' %}". + # That's why we have to check if it's a string literal first. + token = parser.stream.current + if token.test(lexer.TOKEN_STRING): + expr = nodes.Const(token.value, lineno=token.lineno) + next(parser.stream) + else: + expr = parser.parse_expression(False) + + return expr diff --git a/trestle/core/jinja/ext.py b/trestle/core/jinja/ext.py new file mode 100644 index 000000000..09795b118 --- /dev/null +++ b/trestle/core/jinja/ext.py @@ -0,0 +1,40 @@ +# -*- mode:python; coding:utf-8 -*- + +# Copyright (c) 2024 The OSCAL Compass Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Trestle core.jinja extension loading functionality.""" +import functools +import logging +from typing import List + +from trestle.core.jinja import filters, tags +from trestle.core.jinja.base import Extension, TrestleJinjaExtension +from trestle.core.plugins import discovered_plugins + +logger = logging.getLogger(__name__) + + +@functools.cache +def extensions() -> List[Extension]: + """Return list of Jinja extensions packaged with compliance-trestle and included from plugins.""" + extensions = [tags.MDSectionInclude, tags.MDCleanInclude, tags.MDDatestamp, filters.JinjaSSPFilters] + # This block is uncovered as trestle cannot find plugins in it's unit tests - it is the base module. + for plugin, ext_cls in discovered_plugins('jinja_ext'): # pragma: nocover + # add extensions (derived from TrestleJinjaExtension) to extensions list + if issubclass(ext_cls, Extension): + # don't add Extension or TrestleJinjaExtension + if ext_cls is not TrestleJinjaExtension and ext_cls is not Extension: + extensions.append(ext_cls) + logger.info(f'{ext_cls} added to jinja extensions from plugin {plugin}') + return extensions diff --git a/trestle/core/jinja/filters.py b/trestle/core/jinja/filters.py new file mode 100644 index 000000000..574281999 --- /dev/null +++ b/trestle/core/jinja/filters.py @@ -0,0 +1,76 @@ +# -*- mode:python; coding:utf-8 -*- + +# Copyright (c) 2024 The OSCAL Compass Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Trestle utilities to customize jinja filters.""" +import logging +from typing import Any, Iterator, List, Optional + +from jinja2.environment import Environment + +from trestle.common.list_utils import as_list, get_default +from trestle.core.jinja.base import TrestleJinjaExtension +from trestle.oscal.common import Party, ResponsibleParty +from trestle.oscal.ssp import Diagram, SystemSecurityPlan + +logger = logging.getLogger(__name__) + + +def first_or_none(value: Optional[List[Any]]) -> Optional[Any]: + """Retrieve the first array entry, or None for lists that are None or empty.""" + return next(iter(as_list(value)), None) + + +def get_party(uuid: str, ssp: SystemSecurityPlan) -> Optional[Party]: + """Get the metadata.parties entry for this UUID.""" + return next((x for x in as_list(ssp.metadata.parties) if x.uuid == uuid), None) + + +def parties_for_role(responsible_parties: List[ResponsibleParty], role_id: str, + ssp: SystemSecurityPlan) -> Iterator[Party]: + """Get a list of parties from a list of responsible_parties and a given role_id.""" + logger.debug(f'Finding parties for role: {role_id}') + for responsible_party in as_list(responsible_parties): + if responsible_party.role_id == role_id: + logger.debug( + f'Found responsible party for role_id: {role_id} with {len(responsible_party.party_uuids)} parties' + ) + for uuid in responsible_party.party_uuids: + logger.debug(f'Looking for parties with uuid: {uuid}') + party = get_party(uuid, ssp) + if party: + yield party + + +def diagram_href(diagram: Optional[Diagram]) -> str: + """Retrieve the diagrams's link href.""" + if diagram: + return next((link.href for link in as_list(diagram.links) if link.rel == 'diagram'), '') + else: + return '' + + +class JinjaSSPFilters(TrestleJinjaExtension): + """Collection of useful OSCAL-specific filters.""" + + def __init__(self, environment: Environment) -> None: + """Initialize class and add filters.""" + super(JinjaSSPFilters, self).__init__(environment) + + environment.filters['as_list'] = as_list + environment.filters['get_default'] = get_default + environment.filters['first_or_none'] = first_or_none + environment.filters['get_party'] = get_party + environment.filters['parties_for_role'] = parties_for_role + environment.filters['diagram_href'] = diagram_href diff --git a/trestle/core/jinja.py b/trestle/core/jinja/tags.py similarity index 86% rename from trestle/core/jinja.py rename to trestle/core/jinja/tags.py index 3f97e219f..4ce11f725 100644 --- a/trestle/core/jinja.py +++ b/trestle/core/jinja/tags.py @@ -13,18 +13,18 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -"""Trestle utilities to customize .""" +"""Trestle utilities to customize jinja tags.""" import logging from datetime import date import frontmatter -from jinja2 import lexer, nodes +from jinja2 import lexer from jinja2.environment import Environment -from jinja2.ext import Extension from jinja2.parser import Parser from trestle.common import err +from trestle.core.jinja.base import TrestleJinjaExtension from trestle.core.markdown import docs_markdown_node, markdown_const logger = logging.getLogger(__name__) @@ -43,36 +43,6 @@ def adjust_heading_level(input_md: str, expected: int) -> str: return output_md -class TrestleJinjaExtension(Extension): - """Class to define common methods to be inherited from for use in trestle.""" - - # This - max_tag_parse = 20 - - def __init__(self, environment: Environment) -> None: - """Ensure enviroment is set and carried into class vars.""" - super().__init__(environment) - - @staticmethod - def parse_expression(parser): - """Safely parse jinja expression.""" - # Licensed under MIT from: - # https://github.com/MoritzS/jinja2-django-tags/blob/master/jdj_tags/extensions.py#L424 - # Due to how the jinja2 parser works, it treats "foo" "bar" as a single - # string literal as it is the case in python. - # But the url tag in django supports multiple string arguments, e.g. - # "{% url 'my_view' 'arg1' 'arg2' %}". - # That's why we have to check if it's a string literal first. - token = parser.stream.current - if token.test(lexer.TOKEN_STRING): - expr = nodes.Const(token.value, lineno=token.lineno) - next(parser.stream) - else: - expr = parser.parse_expression(False) - - return expr - - class MDSectionInclude(TrestleJinjaExtension): """Inject the parameter of the tag as the resulting content.""" diff --git a/trestle/core/plugins.py b/trestle/core/plugins.py new file mode 100644 index 000000000..80d10b512 --- /dev/null +++ b/trestle/core/plugins.py @@ -0,0 +1,47 @@ +# -*- mode:python; coding:utf-8 -*- + +# Copyright (c) 2024 The OSCAL Compass Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Plugin discovery code.""" + +import importlib +import inspect +import logging +import pathlib +import pkgutil +from typing import Any, Iterator, Tuple + +logger = logging.getLogger(__name__) + +_discovered_plugins = { + name: importlib.import_module(name) + for finder, + name, + ispkg in pkgutil.iter_modules() + if name.startswith('trestle_') +} + + +def discovered_plugins(search_module: str) -> Iterator[Tuple[str, Any]]: + """Yield discovered plugin classes within a given module name.""" + logger.debug(_discovered_plugins) + # This block is uncovered as trestle cannot find plugins in it's unit tests - it is the base module. + for plugin, value in _discovered_plugins.items(): # pragma: nocover + for _, module, _ in pkgutil.iter_modules([pathlib.Path(value.__path__[0], search_module)]): + logger.debug(module) + plugin_module = importlib.import_module(f'{plugin}.{search_module}.{module}') + clsmembers = inspect.getmembers(plugin_module, inspect.isclass) + logger.debug(clsmembers) + for _, plugin_cls in clsmembers: + yield (plugin, plugin_cls)