diff --git a/research/global-config/index.md b/research/global-config/index.md new file mode 100644 index 0000000..e521034 --- /dev/null +++ b/research/global-config/index.md @@ -0,0 +1,281 @@ +--- +title: Global configuration +authors: + - mmassari +sidebar_position: 1 +--- + +# Global configuration styles + +I will analyze 3 approaches for managing configuration relationships: + +- templating +- global configuration + overlays +- inheritance + +# 1. Templating + +## https://github.com/packit/templeates/config/simple-pull-from-upstream.yaml.j2 + +```yml +# Packit pull-from-upstream config +specfile_path: { { specfile_path } } + +upstream_package_name: { { upstream_package_name } } +downstream_package_name: { { downstream_package_name } } +upstream_project_url: { { upstream_package_url } } +upstream_tag_template: v{version} + +jobs: + - job: pull_from_upstream + trigger: release + dist_git_branches: + - fedora-rawhide + - job: koji_build + trigger: commit + allowed_pr_authors: ["packit", { { allowed_pr_authors } }] + dist_git_branches: + - fedora-rawhide +``` + +## https://gitlab.gnome.org/packit/templates/configs/gnome-tests.yaml.j2 + +```yml +# Gnome default tests config +jobs: + - job: tests + trigger: pull_request + packages: [{{ downstream_package_name }}] + tmt_plan: "smoke|full|packit-integration|{{ tmt_other_plans }}" + targets: + - fedora-rawhide + {% if tests_on_commit %} + - job: tests + trigger: commit + packages: [{{ downstream_package_name }}] + tmt_plan: "smoke|full|packit-integration|{{ tmt_other_plans }}" + targets: + - fedora-rawhide + {% endif %} +``` + +## https://gitlab.gnome.org/package/packit.yaml + +```yml +# A gnome package packit config +templates: + - https://github.com/packit/templeates/config/simple-pull-from-upstream.yaml.j2 + vars: + specfile_path: specfile_path + upstream_package_name: upstream_package_name + downstream_package_name: downstream_package_name + upstream_project_url: upstream_project_url + allowed_pr_authors: allowed_pr_authors + - https://gitlab.gnome.org/packit/templates/configs/gnome-tests.yaml.j2 + downstream_package_name: downstream_package_name + tmt_other_plans: tmt_other_plans + tests_on_commit: false +``` + +# 2. Global config + overlay + +## https://github.com/packit/templates/configs/standard-pull-from-upstream.yaml.j2 + +```yml +# Packit pull-from-upstream config +specfile_path: { { specfile_path } } + +upstream_package_name: { { upstream_package_name } } +downstream_package_name: { { downstream_package_name } } +upstream_project_url: { { upstream_package_url } } +upstream_tag_template: v{version} + +jobs: + - job: pull_from_upstream + trigger: release + dist_git_branches: + - fedora-rawhide + - job: koji_build + trigger: commit + allowed_pr_authors: ["packit", { { allowed_pr_authors } }] + dist_git_branches: + - fedora-rawhide +``` + +## https://gitlab.gnome.org/packit/templates/configs/default_packit.yaml.j2 + +```yml +# Gnome default packit config +config: + base: https://github.com/packit/templates/configs/standard-pull-from-upstream.yaml.j2 + values: + allowed_pr_authors: gnome-admins + +jobs: + - job: tests + trigger: pull_request + packages: [{{ downstream_package_name }}] + tmt_plan: "smoke|full|packit-integration|{{ tmt_other_plans }}" + targets: + - fedora-rawhide + {% if tests_on_commit %} + - job: tests + trigger: commit + packages: [{{ downstream_package_name }}] + tmt_plan: "smoke|full|packit-integration|{{ tmt_other_plans }}" + targets: + - fedora-rawhide + {% endif %} +``` + +## https://gitlab.gnome.org/package/packit.yaml + +```yml +# A gnome package packit config +config: + base: https://gitlab.gnome.org/packit/templates/configs/default_packit.yaml.j2 + values: + specfile_path: specfile_path + upstream_package_name: upstream_package_name + downstream_package_name: downstream_package_name + upstream_project_url: upstream_package_url + tmt_other_plans: package-tests + tests_on_commit: false +``` + +# 3. Inheritance + +## https://github.com/packit/templates/configs/standard-pull-from-upstream.yaml + +```yml +# Packit pull-from-upstream config +specfile_path: -OVERRIDE ME- + +upstream_package_name: -OVERRIDE ME- +downstream_package_name: -OVERRIDE ME- +upstream_project_url: -OVERRIDE ME- + +upstream_tag_template: v{version} + +jobs: + - job: pull_from_upstream + trigger: release + dist_git_branches: + - fedora-rawhide + - job: koji_build + trigger: commit + allowed_pr_authors: ["packit"] + dist_git_branches: + - fedora-rawhide +``` + +## https://gitlab.gnome.org/packit/templates/configs/default_packit.yaml + +```yml +# Gnome default packit config +inherit: https://github.com/packit/templates/configs/standard-pull-from-upstream.yaml + +jobs: + - job: koji_build + allowed_pr_authors: ["packit", "gnome-admins"] + + - job: tests + trigger: pull_request + tmt_plan: "smoke|full|packit-integration" + targets: + - fedora-rawhide +``` + +## https://gitlab.gnome.org/package/packit.yaml + +```yml +# A gnome package packit config +inherit: https://gitlab.gnome.org/packit/templates/configs/default_packit.yaml + +specfile_path: specfile_path +upstream_package_name: upstream_package_name +downstream_package_name: downstream_package_name +upstream_project_url: upstream_package_url + +jobs: + - job: tests + packages: ["downstream_package_name"] + tmt_plan: "smoke|full|packit-integration|package-tests" +``` + +# PROs and CONs + +## Templating + +### Pros + +Flexible, probably the most flexible implementation that allows to freely mix configuration snippets for creating a customized final configuration. + +### Cons + +The "pure" templating mechanism, in the above example, requires the package maintainer to know that koji builds, in the gnome ecosystem, should be allowed for any **gnome-admin**, instead _inheritance_ and _global config + overlays_ encapsulate well the knowledge in the middle layer packit config. + +Templating is flexible but on the other end it is more error prone; there is no _base configuration_ and a packager can list templates in the wrong order. + +Probably, in the end, the packager will use smaller config snippets, decreasing performance and readability. + +## Global config + overlays + +### Pros + +Good knowledge encapsulation in middle layers (see the **gnome-admin** for allowed_pr_authors in the above example). + +Explicit and thus easily readable, since the use of templating. + +Flexible, config snippets can easily be removed using template conditional functionalities (as in the above example for the test job with trigger commit). + +### Cons + +The templating syntax can be more error prone if compared with inheritance. + +## Inheritance + +### Pros + +Concise, it's the most concise syntax we could use and probably the least error prone. + +### Cons + +Poor flexibility, I don't see an easy way to disable the above test job with trigger commit. + +Not really explicit, even though we use a placeholder it is harder to recognize the keys that need overriding. + +# Implementation + +Personally I find the _global config + overlays_ approach the best and in this case we would need to: + +- add the following keys to the `PackageConfig` class: + +```yml +config: + base: https://gitlab.gnome.org/packit/templates/configs/default_packit.yaml.j2 + values: ... +``` + +- load the packit.yaml file and search for the `config` key in it. If a `config` key is found we need to **recursively** look for the _base config_ and start processing all the templates in the chain, creating a new temporary packit.yml that will be used instead of the original one. + I see this code tied with the `LocalProject` class but I can be wrong. + We should make the new code work both for the packit CLI and the packit-service. Thinking at packit CLI, we should probably stay flexible and let the `base: URI` be also a local url (like `file:///`). + +- let the user know what the final configuration looks like (both for CLI and service). + +## Jinja2 vs Ansible library + +For template management I would probably just use the **jinja2 template library**, even though we can also think about the ansible library. +The ansible library could let us use `built-in filters and functions` but I don't see use cases for them and as a cons it has a heavier dependency footprint. + +# Performances + +Splitting the configuration in multiple configuration files will lead obviously to worst performance. Personally I don't see a way to prevent it. + +We can limit the number of recursion steps; 3/4 steps are, from my point of view, more than enough. Having a recursion limit will avoid an infinite recursion for malformed configurations. + +# packit-service defaults + +It could happen that the packit-service config defaults for the "Fedora CI instance" and those for the "Usual instance" (as an example) diverge. + +If it happens we could create two _hidden_, _inner_ packit config bases, one per instance, which will always be used for merging any packit config we process; in this way the differences will be grouped explicitly in a single place and we could, probably, enable and disable jobs for one instance or the other (as an example `pull-from-upstream` should not appear in fedora ci packit configuration) just using templating.