basis mol10 with 10.0.2

.gitignore

@@ -1,2 +1,3 @@
 *.tar.gz
 **/.vagrant
+*inventory.ini

molgenis10/.yamllint

@@ -0,0 +1,35 @@
+yaml-files:
+  - '*.yaml'
+  - '*.yml'
+  - '.yamllint'
+
+rules:
+  braces: enable
+  brackets: enable
+  colons: enable
+  commas: enable
+  comments:
+    level: warning
+  comments-indentation:
+    level: warning
+  document-end: disable
+  empty-lines: enable
+  empty-values: disable
+  hyphens: enable
+  indentation: enable
+  key-duplicates: enable
+  key-ordering: disable
+  line-length:
+    max: 140
+    allow-non-breakable-words: true
+    allow-non-breakable-inline-mappings: false
+  new-line-at-end-of-file: enable
+  new-lines: enable
+  octal-values: disable
+  quoted-strings: disable
+  trailing-spaces: enable
+  truthy:
+    allowed-values: ['True', 'true', 'False', 'false', 'yes', 'no']
+    check-keys: true
+    level: warning
+

molgenis10/README.md

@@ -0,0 +1,53 @@
+# Ansible Collection - molgenis.molgenis10
+
+## Migration from Molgenis9 (molgenis 9.1.y)
+Molgenis 10.x.y uses elasticsearch 7 
+
+
+
+## Usage
+
+### Image remote server through ssh and ansible
+'ansible-galaxy collection install molgenis.molgenis10'
+
+You can use ansible to setup up a basic molgenis server on a remote linux machine. At the moment we support the following Linux flavours:
+
+* RedHat/CentOS 7 (EoL June 2024)
+* RedHat Enterprise Linux 8 (EoL May 2029)
+* Ubuntu 20.04 (EoL April 2025)
+* RockyLinux 8 (EoL May 2029)
+
+The RedHat/Rocky servers needs to have selinux enabled (might need a reboot) before your run the playbook.
+It is also recommended to first upgrade/patch your server to the latest patch level, before running the playbook.
+
+You need to have setup an ssh trust between the system your run ansible on and the remote linux server root account where molgenis will be installed.
+
+First create an inventory.ini file with the FQDNs/ip addresses of your target hosts. Then execute the next command (from the `molgenis10` directory):
+
+'ansible-playbook playbook.yml -e memory="4"' 
+
+We currently support automatic memory configuration for 4, 8 and 16 Gb of memory. This way tomcat/java and elasticsearch each get a decent amount of memory for Molgenis to run.
+
+
+To upgrade a remote molgenis server to a newer molgenis version within 10.x.y versioin range, you can run:
+
+'ansible-playbook playbook_upgrade.yml'
+
+To upgrade a molgenis 9.x.y server to molgenis 10.x.y, you can run:
+
+'ansible-playbook playbook_upgrade_from_9.yml -e memory="4"'
+
+
+### Usage locally for testing purposes
+You can use Vagrant to boot up the image. Please execute (from the `molgenis10` directory):
+
+`vagrant up`
+
+When updating the playbook
+`vagrant up --provision`
+
+When running a specific version of and OS:
+
+`VAGRANT_VAGRANTFILE=Vagrantfile.centos7 vagrant up`
+
+Access the instance from `http://localhost:8080` in the browser.

molgenis10/Vagrantfile

@@ -0,0 +1,14 @@
+Vagrant.configure("2") do |config|
+  config.vm.box = "centos/8"
+  config.vm.box_version = "1905.1"
+  config.vm.network "forwarded_port", guest: 80, host: 8080
+  config.vm.provider "virtualbox" do |vb|
+    vb.gui = false
+    vb.customize ['modifyvm', :id, '--graphicscontroller', 'none']
+    vb.customize ['modifyvm', :id, '--audio', 'none']
+  end
+  config.vm.provision "ansible" do |ansible|
+    ansible.limit = "all"
+    ansible.playbook = "playbook.yml"
+  end
+end

molgenis10/collections/requirements.yml

@@ -0,0 +1,12 @@
+---
+collections:
+  # Install a collection from Ansible Galaxy.
+  - name: ansible.posix
+    version: 1.1.1
+    source: https://galaxy.ansible.com
+  - name: community.general
+    version: 2.0.1
+    source: https://galaxy.ansible.com
+  - name: community.postgresql
+    version: 1.6.0
+    source: https://galaxy.ansible.com

molgenis10/docs/README.md

@@ -0,0 +1,47 @@
+# How to run your own Molgenis instance
+
+It is possible to run your Molgenis in docker containers to test if the software fits your needs [our docker image](https://github.com/molgenis/docker), but we recommend to run serious work in an VM based Molgenis instance. As the Molgenis support team we host 100+ of these instances for research projects in our own OpenStack VMs. But some research projects needs to keep their data at their own site. For these request we did not have an easy solution. We use ansibe playbooks to setup, maintain and secure our VMs, but these playbooks contain lots of RUG/UMCG specific infrastructure taks and therefor we put them in a private GitHub repo.
+
+This Ansible Galaxy collection is a subset of the roles used in our private repo, and allows easy setup of your own basic molgenis instance on your own infrastrcuture.
+
+
+### Software components that are part of the Molgenis instance:
+- Java
+- Tomcat
+- NGinX
+- Minio
+- Postgresql
+- ElasticSearch
+
+## Requirements
+- OS: RedHat 8 or Centos 8 or Rocky8 of Ubuntu 20.04 minimal install
+- Hardware specs:  
+  - 4-8 Gb memory
+  - 2-4 cores
+  - 25 Gb diskspace
+- ssh access to server/VM with ssh keys (root or unprivileged user)
+- Ansibe or AnsibleTower/Ansible AWX
+
+## Ansible
+We chose Ansible for our orchestration because it is lightwight and only needs python and ssh.
+
+We use Ansible AWX (the free Ansible Tower) to install, patch, upgrade our 100+ VMs with Molgenis. But for smaller setups, plain ansible from a laptop will also do nicely.
+
+## Ansible Basic playbook
+- sets up basic Molgenis on http.
+- Modular setup with Roles
+- easily extendable with pre an post roles to setup customer/cloud specifics
+  - SSL certs
+  - firewall make sure you open 80/443
+  - backup
+  - monitoring/logging
+
+Needed extras to fit custom install:
+- SSL tcp/443
+
+
+## Ansible Galaxy collection
+The ansible playbook and roles for a minimum install of Molgenis can be found in `http://galaxy.ansible.com/molgenis/molgenis10`
+
+The ansible scripts used to install Molgenis in the RUG OpenStack are in a private GitHub repo, but will be adapted in the future to also use the basic roles provided in the Ansible Galaxy collection.
+

molgenis10/galaxy.yml

@@ -0,0 +1,59 @@
+### REQUIRED
+
+# The namespace of the collection. This can be a company/brand/organization or product namespace under which all
+# content lives. May only contain alphanumeric characters and underscores. Additionally namespaces cannot start with
+# underscores or numbers and cannot contain consecutive underscores
+namespace: molgenis
+
+# The name of the collection. Has the same character restrictions as 'namespace'
+name: molgenis10
+
+# The version of the collection. Must be compatible with semantic versioning
+version: 10.0.1
+
+# The path to the Markdown (.md) readme file. This path is relative to the root of the collection
+readme: README.md
+
+# A list of the collection's content authors. Can be just the name or in the format 'Full Name <email> (url)
+# @nicks:irc/im.site#channel'
+authors:
+- Erik Schaberg <[email protected]>
+- Sido Haakma <[email protected]>
+- Molgenis operations team <[email protected]>
+
+
+### OPTIONAL but strongly recommended
+
+# A short summary description of the collection
+description: this collection contains roles and playbooks to roll-out and update a Molgenis instance in your own infrastructure (see https://www.molgenis.org)
+
+# Either a single license or a list of licenses for content inside of a collection. Ansible Galaxy currently only
+# accepts L(SPDX,https://spdx.org/licenses/) licenses. This key is mutually exclusive with 'license_file'
+license:
+- GPL-2.0-or-later
+
+# The path to the license file for the collection. This path is relative to the root of the collection. This key is
+# mutually exclusive with 'license'
+license_file: ''
+
+# A list of tags you want to associate with the collection for indexing/searching. A tag name has the same character
+# requirements as 'namespace' and 'name'
+tags: [molgenis]
+
+# Collections that this collection requires to be installed for it to be usable. The key of the dict is the
+# collection label 'namespace.name'. The value is a version range
+# L(specifiers,https://python-semanticversion.readthedocs.io/en/latest/#requirement-specification). Multiple version
+# range specifiers can be set and are separated by ','
+dependencies: {}
+
+# The URL of the originating SCM repository
+repository: https://github.com/molgenis/
+
+# The URL to any online docs
+documentation: https://www.molgenis.org/learn.html 
+
+# The URL to the homepage of the collection/project
+homepage: https://molgenis.org
+
+# The URL to the collection issue tracker
+issues: https://github.com/molgenis/molgenis/issues

molgenis10/meta/runtime.yml

@@ -0,0 +1 @@
+requires_ansible: ">=2.9,<2.10"

molgenis10/playbook.yml

@@ -0,0 +1,40 @@
+---
+- name: Deployment for MOLGENIS - operational development
+  hosts: "{{ host_group }}"
+  gather_facts: true
+  become: true
+  vars:
+    host_group: "all"
+    experimental: false
+    ci: false
+    minio:
+      access_key: molgenis
+      secret_key: molgenis
+      port: 9000
+      host: http://localhost
+    molgenis:
+      setup: true
+      version: 10.0.2
+      ssh: true
+
+  roles:
+    # system configuration
+    - role: swap
+    - role: preinstall
+    # application configuration
+    - role: elasticsearch_7
+    - role: postgres_11
+    - role: minio
+      vars:
+        version: 2020-07-02T00-15-09Z
+        data: /var/lib/minio/data
+        access_key: "{{ minio.access_key }}"
+        secret_key: "{{ minio.secret_key }}"
+    - role: tomcat
+    - role: molgenis_user
+    - role: molgenis_application
+    - role: nginx
+
+    # additional configuration
+    - role: python
+    - role: postinstall

molgenis10/playbook_upgrade.yml

@@ -0,0 +1,92 @@
+---
+- name: Upgrade molgenis to 10.x.y on centos 7, Rocky 8 and Ubuntu 20.04
+  hosts: "{{ host_group }}"
+  gather_facts: true
+  become: true
+  vars:
+    host_group: "all"
+    molgenis:
+      setup: true
+      version: 10.0.2
+
+  tasks:
+    - name: Gathering package facts
+      package_facts:
+        manager: auto
+
+    - name: Check if war of correct version is present
+      ansible.builtin.stat:
+        path: /usr/local/share/molgenis/war/molgenis-app-{{ molgenis.version }}.war
+      register: war_version
+
+    - name: Stop tomcat
+      ansible.builtin.systemd:
+        name: tomcat
+        state: stopped
+      when: war_version.stat.isreg is not defined
+
+    - name: Stop elasticsearch
+      ansible.builtin.systemd:
+        name: elasticsearch
+        state: stopped
+      when: war_version.stat.isreg is not defined
+
+    - name: Remove old ROOT.wars if rpm was present
+      ansible.builtin.file:
+        state: absent
+        path: "{{ item }}"
+      with_items:
+      - /usr/share/tomcat/webapps/ROOT.war
+      - /usr/local/share/molgenis/war/ROOT.war
+      when:
+        - '"molgenis" in ansible_facts.packages'
+
+    - name: Remove old molgenis war folder
+      ansible.builtin.file:
+        path: /usr/local/share/molgenis/war
+        state: absent
+      when: war_version.stat.isreg is not defined
+
+    - name: Create empty molgenis war folder
+      ansible.builtin.file:
+        path: /usr/local/share/molgenis/war
+        state: directory
+        owner: molgenis
+        group: molgenis
+        mode: '0775'
+      when: war_version.stat.isreg is not defined
+
+    - name: install war MOLGENIS
+      ansible.builtin.get_url:
+        url: https://registry.molgenis.org/repository/maven-public/org/molgenis/molgenis-app/{{ molgenis.version }}/molgenis-app-{{ molgenis.version }}.war
+        mode: '0660'
+        owner: molgenis
+        group: molgenis
+        dest: /usr/local/share/molgenis/war/
+      when: war_version.stat.isreg is not defined
+
+    - name: create symbolic link
+      ansible.builtin.file:
+        src: /usr/local/share/molgenis/war/molgenis-app-{{ molgenis.version }}.war
+        dest: /usr/share/tomcat/webapps/ROOT.war
+        owner: molgenis
+        group: molgenis
+        state: link
+      when: war_version.stat.isreg is not defined
+
+    - name: role nginx
+      ansible.builtin.include_role:
+        name: nginx
+      when: war_version.isreg is not defined
+
+    - name: Start elasticsearch
+      ansible.builtin.systemd:
+        name: elasticsearch
+        state: started
+      when: war_version.isreg is not defined
+
+    - name: Start tomcat
+      systemd:
+        name: tomcat
+        state: started
+      when: war_version.isreg is not defined