From e3a4ba71801f2e562c8f9cc831325c14207e51e1 Mon Sep 17 00:00:00 2001 From: Alexandre Possebom Date: Thu, 25 Jul 2019 05:52:29 -0300 Subject: [PATCH] Move documentation to Read the Docs. (#287) This documentation below is centralized in RTD: - Best practices - Dependencies - Howto Inhibit Upgrade - Tests --- docs/README.md | 3 + docs/best-practices.md | 153 ------------------------------- docs/dependencies.md | 56 ------------ docs/howto-inhibit-upgrade.md | 167 ---------------------------------- docs/tests.md | 22 ----- 5 files changed, 3 insertions(+), 398 deletions(-) create mode 100644 docs/README.md delete mode 100644 docs/best-practices.md delete mode 100644 docs/dependencies.md delete mode 100644 docs/howto-inhibit-upgrade.md delete mode 100644 docs/tests.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000000..2a0d966390 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,3 @@ +# Leapp repository documentation + +The Leapp repository documentation has been moved to [Read the Docs](https://leapp.readthedocs.io/). diff --git a/docs/best-practices.md b/docs/best-practices.md deleted file mode 100644 index a9044a4f0a..0000000000 --- a/docs/best-practices.md +++ /dev/null @@ -1,153 +0,0 @@ -# Best practices for writing actors - -## Follow the contribution guidelines - -See the [contribution guidelines for leapp-repository](../CONTRIBUTING.md). - -## Avoid running code on a module level - -Despite the actors being written in Python, they are loaded beforehand to get all the meta-information from them. To -avoid slow downs during this load phase we ask you to not execute any code that is not absolutely necessary -on a module level. Simple, fast code with no side-effects is acceptable, for example constant definitions. - -## Avoid certain global imports - -On a module level, try to import only the Python Standard Library modules or modules provided by the Leapp framework. -Import any other module within your function or method. This has to do with possible slow down when loading repositories -by the leapp tool, and also to avoid unnecessary complications for our tests automation which needs to inspect the -actors beforehand. - -## Use the snactor tool for development - -The snactor tool helps you with creating the base layout of a new Leapp repository, and with creating boilerplates for -the repository artifacts like actors, models, tags, topics, and workflows. It also helps you with debugging as it is -able to execute individual actors. - -See the [tutorial on basic usage of snactor](https://leapp.readthedocs.io/en/latest/first-actor.html). - -## Move generic functionality to libraries - -Part of your actor's functionality might end up being rather generic or abstract. In that case, consider converting it -to a shared library function. You can introduce it in one of these two places: - -- The [Leapp Standard Library](https://leapp.readthedocs.io/en/latest/pydoc/leapp.libraries.stdlib.html) - - The most generic functionality which actors of any workflow can use, e.g. function for exectuting a shell command, - should go to the [Leapp Standard Library](https://leapp.readthedocs.io/en/latest/pydoc/leapp.libraries.stdlib.html). - For that, please submit proposals through issues or pull requests under the - [leapp GitHub repository](https://github.com/oamg/leapp/). - -- Leapp repository-level shared library - - The functionality that may be useful for other actor developers but its scope is limited to the workflow you are - writing your actor for, e.g. for an OS in-place upgrade workflow, should go to the `/libraries` - folder. In this case, we welcome your proposals under the - [leapp-repository on GitHub](https://github.com/oamg/leapp-repository). - -## Discover standard library functions - -Before implementing functionality for your actor, browse through the available functionality provided by: - -- the [Leapp Standard Library](https://github.com/oamg/leapp/tree/master/leapp/libraries/stdlib/), -- the shared library of your Leapp repository (`/libraries` folder). - -These libraries contain functions that may satisfy your needs. Using them can save you time, lower code complexity and -help avoiding duplicate code. Improvement proposals for the library functions are welcome. - -## Prefer using stdlib functions over shell commands - -Sources of external functionality to be used in your actor in order of preference: - -1. the [Leapp Standard Library](https://github.com/oamg/leapp/tree/master/leapp/libraries/stdlib/) -2. the [Python Standard Library](https://docs.python.org/3/library/index.html) -3. shell commands - -Examples: - -- Prefer `os.symlink` over `ls -s` -- Prefer `os.remove` over `rm` - -There might be a valid reason for calling a shell command instead of a standard library function, e.g. the Python's -`shutil.copyfile` is not able to retain all of the file attributes, as opposed to shell's `cp --preserve`. - -## Utilize messages produced by existing actors - -As with the Leapp Standard Library mentioned above, it may be beneficial for you to skim through the actors already in -place in the [leapp-repository](https://github.com/oamg/leapp-repository). You might be interested in the messages they -produce, for example: - -- [SystemFactsActor](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/actors/systemfacts/actor.py) - - information about kernel modules, yum repos, sysctl variables, users, firewall, SELinux, etc. -- [OSReleaseCollector](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/actors/osreleasecollector/actor.py) - - system release information -- [RpmScanner](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/actors/rpmscanner/actor.py) - - list of installed packages -- [StorageScanner](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/actors/storagescanner/actor.py) - - storage information - -In case you find any message of the existing actors to be incorrect, incomplete or misleading, we encourage you to -improve the respective actors. - -## Write unit testable code - -Write all the actor’s logic in the actor’s private library in order to be able to write unit tests for each of the -function. It is not currently possible to unit test any method or function in the _actor.py_. Then, ideally, the -`actor.process()` should contain only calling an entry point to the actor's library. To create an actor’s library, -create _libraries_ folder in the actor’s folder and in there create an arbitrarily named python file, e.g. _library.py_. - -_myactor/actor.py_: - -```python -from leapp.libraries.actor.library import do_the_actor_thingy - -class MyActor(Actor): - # - def process(self): - do_the_actor_thingy(self) -``` - -_myactor/libraries/library.py_: - -```python -def do_the_actor_thingy(actor): - actor.log.debug("All the actor’s logic shall be outside actor.py") -``` - -For more about unit testing, see the [tutorial](https://leapp.readthedocs.io/en/latest/unit-testing.html). - -## Do not introduce new dependencies - -Ideally, actors shouldn't require any additional dependency on top of the dependencies already in the -[leapp](https://github.com/oamg/leapp/blob/master/packaging/leapp.spec) and -[leapp-repository](https://github.com/oamg/leapp-repository/blob/master/packaging/leapp-repository.spec) spec files, -which are, as of December 2018, just these: - -- dnf -- python-six -- python-setuptools -- findutils - -Rather than adding a new dependency to the spec file, detect in the actor if the package is installed and -if not, have a fallback option or skip the action you wanted to perform and report to the user that they should install -the package if they want to experience as smooth upgrade as possible. - -If you're writing an actor for the RHEL 7 to RHEL 8 in-place upgrade workflow and you really need to have some package -installed, then the only acceptable packages to depend on are the the ones available in the minimal installation of -RHEL 7 (packages from the @core group + their dependencies). - -For more details about dependencies and how to modify them, see the [How to deal with dependencies](https://github.com/oamg/leapp-repository/blob/master/docs/dependencies.md) - -## Use convenience exceptions to stop the actor's execution - -If you encounter unexpected input or any other error during execution of an actor and want to stop it, use these exceptions: - -- [StopActorExecution](https://leapp.readthedocs.io/en/latest/pydoc/leapp.html#leapp.exceptions.StopActorExecution) - raising this exception is a convenient to stop actor's execution with no side effect - it has the same effect as returning `None` from the main `process()` method in the `actor.py` -- [StopActorExecutionError](https://leapp.readthedocs.io/en/latest/pydoc/leapp.html#leapp.exceptions.StopActorExecutionError) - raising this exception will stop actor's execution and notify the framework that an error has occurred and can influence the result of the workflow execution. - -In case of [StopActorExecutionError](https://leapp.readthedocs.io/en/latest/pydoc/leapp.html#leapp.exceptions.StopActorExecutionError) the execution of the workflow will stop or not according to the [policy](https://leapp.readthedocs.io/en/latest/pydoc/leapp.workflows.html?highlight=FailPhase#module-leapp.workflows.policies) defined in the workflow, there are three possibilities: - -- [FailImmediately](https://leapp.readthedocs.io/en/latest/pydoc/leapp.workflows.html?highlight=FailPhase#leapp.workflows.policies.Policies.Errors.FailImmediately) - end the workflow execution right away -- [FailPhase](https://leapp.readthedocs.io/en/latest/pydoc/leapp.workflows.html?highlight=FailPhase#leapp.workflows.policies.Policies.Errors.FailPhase) - end the workflow execution after finishing the current phase -- [ReportOnly](https://leapp.readthedocs.io/en/latest/pydoc/leapp.workflows.html?highlight=FailPhase#leapp.workflows.policies.Policies.Errors.ReportOnly) - do not end the workflow execution at all and continue with logging the issue only - -You can also use the [StopActorExecution](https://leapp.readthedocs.io/en/latest/pydoc/leapp.html#leapp.exceptions.StopActorExecution) and [StopActorExecutionError](https://leapp.readthedocs.io/en/latest/pydoc/leapp.html#leapp.exceptions.StopActorExecutionError) exceptions inside a private or shared library. diff --git a/docs/dependencies.md b/docs/dependencies.md deleted file mode 100644 index 8a2b4dca5c..0000000000 --- a/docs/dependencies.md +++ /dev/null @@ -1,56 +0,0 @@ -# How to deal with dependencies in leapp-repository - -First, read this -[document](https://github.com/oamg/leapp/blob/master/docs/source/dependencies.md) -to better understand the difficulties related to package dependencies in the -Leapp project. - -When talking about RHEL 7 to RHEL 8 upgrade, the goal is to cover dependencies -of all Leapp project-related packages, including the leapp-repository packages, -for both RHEL 7 and RHEL 8. Since the situation with dependencies of the leapp -packages is similar to the situation with the leapp-repository dependencies, -this document focuses on the leapp-repository specifics only. - -Currently there are two SPEC files for leapp-repository: - -- The -[leapp-repository.spec](https://github.com/oamg/leapp-repository/blob/master/packaging/leapp-repository.spec) -file is used to build leapp-repository packages and their dependency -metapackage _leapp-repository-deps_ **for RHEL 7**. -- The -[leapp-el7toel8-deps.spec](https://github.com/oamg/leapp-repository/blob/master/packaging/leapp-el7toel8-deps.spec) -file is used to build dependency metapackages _leapp-deps-el8_ and -_leapp-repository-deps-el8_ **for RHEL 8** whose purpose is to replace the -RHEL 7 dependency metapackages _leapp-deps_ and _leapp-repository-deps_ during -the upgrade. - -## What to do in leapp-repository when dependencies of leapp change? - -Go to the section below the line `%package -n %{ldname}` in the -[leapp-el7toel8-deps.spec](https://github.com/oamg/leapp-repository/blob/master/packaging/leapp-el7toel8-deps.spec). -This section creates the RHEL 8 _leapp-deps-el8_ metapackage that replaces the -RHEL7 _leapp-deps_ metapackage. So when the leapp package dependencies change -in the [leapp.spec](https://github.com/oamg/leapp/blob/master/packaging/leapp.spec) -together with incrementing version of the **leapp-framework-dependencies** -capability, it's necessary to: - -- provide the same **leapp-framework-dependencies** capability version - by _leapp-deps-el8_ - -- decide if this dependency change also applies to RHEL 8 and if so, update - the dependencies of the _leapp-deps-el8_ metapackage accordingly. - -There can be another case when we need to modify dependencies of leapp on -RHEL 8, e.g. when a RHEL 7 package is renamed or split on RHEL 8. In such case -we don't need to modify the capability value, just update dependencies of the -_leapp-deps-el8_ metapackage, commenting it properly. Nothing else. - -## What to do when leapp-repository dependencies need to change? - -When you want to modify *outer dependencies* of leapp-repository packages, do -that similarly to instructions related to Leapp packages, following the same -rules. Just take care of the **leapp-repository-dependencies** capability -instead of the *leapp-framework-dependencies* capability. Everything else is -the same. Interesting parts of the SPEC files are highlighted in the same way -as described in the -[leapp dependencies document](https://github.com/oamg/leapp/blob/master/docs/source/dependencies.md). \ No newline at end of file diff --git a/docs/howto-inhibit-upgrade.md b/docs/howto-inhibit-upgrade.md deleted file mode 100644 index ec3f73b48e..0000000000 --- a/docs/howto-inhibit-upgrade.md +++ /dev/null @@ -1,167 +0,0 @@ -# Upgrade Inhibition -### Process Inhibition -With latest changes on Leapp and with new actors added to the el7toel8 Leapp -repository, any actor can inhibit the upgrade process by producing a specific -message when a problem is found. The message model to use in this case is -[Report](https://leapp.readthedocs.io/en/latest/pydoc/leapp.reporting.html#leapp.reporting.Report). -If there is at least one Report message with `'inhibitor'` flag produced before -the Report phase, the upgrade will be stopped in the Reports phase, in which the -messages are being collected. It means that any Report message produced -**after** the Report phase will **not** have inhibiting effect. The details -mentioned in the Report messages will be part of the report available to the -user to review. - - -### Sample Actor -Let’s start with a very simple actor that will verify if system architecture is -supported: - -```python -import platform - -from leapp.actors import Actor -from leapp.tags import ChecksPhaseTag - - -class CheckSystemArch(Actor): - """ - Check if system is running at a supported archtecture. If no, inhibit the upgrade process. - - Base on collected system facts, verify if current archtecture is supported, otherwise produces - a message to inhibit upgrade process - """ - - name = 'check_system_arch' - consumes = () - produces = () - tags = (ChecksPhaseTag,) - - def process(self): - if platform.machine() != 'x86_64': - self.log.info("Unsupported arch!") -``` - -If this actor is executed using `snactor` tool in a system with unsupported -architecture, we will see the following output on log: - -```sh -$ snactor run CheckSystemArch --verbose -2019-04-16 15:08:59.622 INFO PID: 1996 leapp: Logging has been initialized -2019-04-16 15:08:59.638 INFO PID: 1996 leapp.repository.sandbox: A new repository 'sandbox' is initialized at /home/leapp/sandbox -2019-04-16 15:08:59.695 INFO PID: 2021 leapp.actors.check_system_arch: Unsupported arch! -``` - -If, instead of only adding a message to the log, the actor writer wants to make -sure that the upgrade process will be stopped in case of unsupported arch, the -actor needs to produce a [Report](https://leapp.readthedocs.io/en/latest/pydoc/leapp.reporting.html#leapp.reporting.Report) -message using one of the `report_*` functions from the [reporting](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/libraries/reporting.py) -shared library with `'inhibitor'` flag. - -```python -import platform - -from leapp.actors import Actor -from leapp.reporting import Report -from leapp.libraries.common.reporting import report_generic -from leapp.tags import ChecksPhaseTag - - -class CheckSystemArch(Actor): - """ - Check if system is running at a supported archtecture. If no, inhibit the upgrade process. - - Base on collected system facts, verify if current archtecture is supported, otherwise produces - a message to inhibit upgrade process - """ - - name = 'check_system_arch' - consumes = () - produces = (Report,) - tags = (ChecksPhaseTag,) - - def process(self): - if platform.machine() != 'x86_64': - report_generic( - title='Unsupported arch', - summary='Upgrade process is only supported on x86_64 systems.', - severity='high', - flags=['inhibitor']) -``` - -Running the actor again, it is possible to verify that a new message was -generated. We will still use `snactor` tool to run the actor, but passing -`--print-output` this time to output all generated messages by the actor: - -```sh -$ snactor run CheckSystemArch --verbose --print-output -2019-04-16 15:20:32.74 INFO PID: 2621 leapp: Logging has been initialized -2019-04-16 15:20:32.94 INFO PID: 2621 leapp.repository.sandbox: A new repository 'sandbox' is initialized at /home/leapp/sandbox -[ - { - "stamp": "2019-04-16T15:20:32.143709Z", - "hostname": "leapp", - "actor": "check_system_arch", - "topic": "system_info", - "context": "904b0170-cfe7-4217-81d3-a259550e73c1", - "phase": "NON-WORKFLOW-EXECUTION", - "message": { - "hash": "dcdf1679b6fd4c2e21bc4e4ed6585df75cd46aeea90a53ca76f469a2a1aa50d2", - "data": "{\"audience\": [\"sysadmin\"], \"detail\": \"{\\\"summary\\\": \\\"Upgrade process is only supported on x86_64 systems.\\\"}\", \"flags\": [\"inhibitor\"], \"renderers\": {\"html\": \"

{{ title }}

{{ summary }}

\", \"plaintext\": \"{{ title }}\\n{{ summary }}\"}, \"severity\": \"high\", \"title\": \"Unsupported arch\"}" - }, - "type": "Inhibitor" - } -] -``` - -Or to inspect closely the message.data filed, we could use `jq` tool: -```sh -snactor run CheckSystemArch --verbose --print-output | jq '.[] | .message.data | fromjson' -{ - "audience": [ - "sysadmin" - ], - "detail": "{\"summary\": \"Upgrade process is only supported on x86_64 systems.\"}", - "flags": [ - "inhibitor" - ], - "renderers": { - "html": "

{{ title }}

{{ summary }}

", - "plaintext": "{{ title }}\n{{ summary }}" - }, - "severity": "high", - "title": "Unsupported arch" -} -``` - -This is all that an actor needs to do in order to verify if some condition is -present on the system and inhibit the upgrade process based on that check. - -After all the system checks are executed by different actors, an existing actor -named [VerifyCheckResults](https://github.com/oamg/leapp-repository/tree/master/repos/system_upgrade/el7toel8/actors/verifycheckresults) -is scheduled to run in the Leapp upgrade workflow. If some [Report](https://leapp.readthedocs.io/en/latest/pydoc/leapp.reporting.html#leapp.reporting.Report) -message with `'inhibitor'` flag was generated by some previous execution of -another actor in any previous phase of the workflow, like the sample one we just -wrote, the following output will be displayed to the user: - -```sh -$ leapp upgrade -(...) -2019-04-16 15:36:54.696 INFO PID: 7455 leapp.workflow: Starting phase Reports -2019-04-16 15:36:54.715 INFO PID: 7455 leapp.workflow.Reports: Starting stage Before of phase Reports -2019-04-16 15:36:54.764 INFO PID: 7455 leapp.workflow.Reports: Starting stage Main of phase Reports -2019-04-16 15:36:54.788 INFO PID: 7455 leapp.workflow.Reports: Executing actor verify_check_results -2019-04-16 15:36:54.854 INFO PID: 8510 leapp.workflow.Reports.verify_check_results: Generated report at /var/log/leapp-report.txt -2019-04-16 15:36:54.923 INFO PID: 7455 leapp.workflow.Reports: Starting stage After of phase Reports -2019-04-16 15:36:54.970 INFO PID: 7455 leapp.workflow: Workflow interrupted due to the FailPhase error policy - -============================================================ - ERRORS -============================================================ - -2019-04-16 15:36:54.871634 [ERROR] Actor: verify_check_results Message: Unsupported arch -2019-04-16 15:36:54.888818 [ERROR] Actor: verify_check_results Message: Ending process due to errors found during checks, see /var/log/leapp-report.txt for detailed report. - -============================================================ - END OF ERRORS -============================================================ -``` diff --git a/docs/tests.md b/docs/tests.md deleted file mode 100644 index 0f363b1abc..0000000000 --- a/docs/tests.md +++ /dev/null @@ -1,22 +0,0 @@ -# Tests for actors - -The Leapp actors are covered by three types of tests - unit, component and e2e. - -## Unit and component tests -- Both unit and component tests are to be placed in the actor's _tests_ folder. -- Tutorial on how to write unit and component tests: https://leapp.readthedocs.io/en/latest/unit-testing.html - -### Unit tests - - These tests deal with individual actor's functions/methods. - - It's not possible to unit test any method/function within the *actor.py*. You can write unit tests only for functions/methods within the actor's libraries. - - Thus, to be able to write unit tests for an actor, ideally the only thing in the _actor.py_'s _process()_ method is calling the entry-point function of the actor's library python module. - - Example of unit tests: https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/actors/checkbootavailspace/tests/unit_test.py - -### Component tests - - These tests provide fabricated input messages for the actor, check the outputs stated in the actor's description. - - These tests should not be written based on the actor's code but rather based on the behavior stated in the actor's description. They could be written by somebody who didn't write the code. - - Example of component tests: https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/actors/checknfs/tests/test_checknfs.py - -## End to end (e2e) tests - - - The QA team for the OAMG projects maintains an internal testing framework facilitating e2e tests.