Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add guide docs for rookify #688

Merged
merged 17 commits into from
Nov 29, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
feat: add guide docs for rookify
Signed-off-by: Boekhorst <[email protected]>
boekhorstb1 committed Oct 8, 2024
commit e86dfda5cffa63e07bf9160ca4895cbc42767772
136 changes: 136 additions & 0 deletions docs/guides/configuration-guide/rookify.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
---
sidebar_label: Configure Rookify: Migrate to Rook from Ceph Ansible (technical preview)
sidebar_position: 31
---

# Configure Rookify: Migrate from Ceph Ansible to Rook (Technical Preview)

:::warning

Rookify is developed to migrate from Ceph-Ansible to Rook _in place_ and _without downtime_.
Nevertheless, it is **strongly advised** to test Rookify in a controlled environment beforehand, such as the [OSISM testbed](https://github.com/osism/testbed). Additionally, ensure that all precautionary backups are taken, and any other necessary safety measures are in place.

:::

The [Rookify GitHub repository](https://github.com/SovereignCloudStack/rookify) includes a README.md that provides a condensed summary of the information covered here.

## Config.yaml

The primary configuration file for Rookify is `config.yaml`. The repository contains an example file for general use, as well as one specifically tailored to the OSISM testbed setup:

- [config.example.yaml](https://github.com/SovereignCloudStack/rookify/blob/main/config.example.yaml)
- [config.example.osism.yaml](https://github.com/SovereignCloudStack/rookify/blob/main/config.example.osism.yaml)

### Parameters

#### General

```yaml title="config.example.yaml"
general:
machine_pickle_file: data.pickle
```

The general section allows for optional definition of a pickle file, which allows for saving the state of the migration as serialized objects on disk. The pickle filed can be named as pleased.

#### Logging

```yaml title="config.example.yaml"
logging:
level: INFO # level at which logging should start
format:
time: "%Y-%m-%d %H:%M.%S" # other example: "iso"
renderer: console # or: json
```

The `logging` section allows for specification of `structlog`. The `level` parameter can be set to all python [log-levels](https://docs.python.org/3/library/logging.html#logging-levels), i.e. `NOTSET, DEBUG, INFO, WARNING, ERROR, CRITICAl`, but it is recommended to use `INFO`.

#### Ceph

```yaml title="config.example.yaml"

ceph:
config: ./.ceph/ceph.conf
keyring: ./.ceph/ceph.client.admin.keyring
```

The `ceph` section specifies the `ceph.conf` and `ceph.client.admin.keyring` of the target systems (i.e. the system where Ceph-Ansible needs to be migrated to Rook).

#### SSH

```yaml title="config.example.yaml"
# fill in correct path to private key
ssh:
private_key: /home/USER/.ssh/cloud.private
hosts:
testbed-node-0:
address: 192.168.16.10
user: dragon
testbed-node-1:
address: 192.168.16.11
user: dragon
testbed-node-2:
address: 192.168.16.12
user: dragon
```

The `ssh` section requires specification of the `private_key` and `hosts`. The `hosts` section specifies the hostnames or aliases (e.g. keys like `testbed-node-0`), their ip-addresses (e.g. if rookify connects to the target systems per VPN add ips starting with `192.186...`) and user for login. If you are using the OSISM testbed, make sure that the private key does not contain any `EOF` or other strings, i.e. make sure the keys are 'clean' to avoid connection errors.

#### Kubernetes

```yaml title="config.example.yaml"

kubernetes:
config: ./k8s/config
```

The `kubernetes` section specifies the kubernetes configuration (e.g. if you use kubectl it is located in `~/.kube/config`) for rookifies kubernetes library. Rookify needs to connect to the kubernetes cluster on the target systems in order to use Rook.

```yaml title="config.example.yaml"
rook:
cluster:
name: osism-ceph
namespace: rook-ceph
ceph:
image: quay.io/ceph/ceph:v18.2.1
```

The `rook` sections requires some information about the Rook installation on the target system. Concerning the `cluster` Rookify needs to know the cluster-name and cluster-namespace. Rookify also needs to know the ceph version used by Rookify, i.e. the image version of the ceph container.

_Note_: Rookify does not install Rook for you. You need to provide a running Rook, i.e. a Rook Operator, on you target system.

_Note_: For OSISM specific migrations, Rookify needs to have some additional information, i.e. the respective labels for the rook resources:

```yaml title="config.example.osism.yaml"
rook:
cluster:
name: osism-ceph
namespace: rook-ceph
mds_placement_label: node-role.osism.tech/rook-mds
mgr_placement_label: node-role.osism.tech/rook-mgr
mon_placement_label: node-role.osism.tech/rook-mon
osd_placement_label: node-role.osism.tech/rook-osd
rgw_placement_label: node-role.osism.tech/rook-rgw
ceph:
image: quay.io/ceph/ceph:v18.2.1
```


#### Migration_modules

```yaml title="config.example.yaml"
migration_modules:
- analyze_ceph
- create_rook_cluster
- migrate_mons
- migrate_osds
- migrate_osd_pools
- migrate_mds
- migrate_mds_pools
- migrate_mgrs
- migrate_rgws
- migrate_rgw_pools
```

Rookify is designed to use a modular structure. It has contains various modules to migrate parts of Ceph-Ansible to Rook. The `migration_modules` section specifies which modules need to be run for the migration. Rookify contains more modules, take a look at the [`src/rookify/module`](https://github.com/SovereignCloudStack/rookify/tree/main/src/rookify/modules) directory to see the ones that are currently implemented.
boekhorstb1 marked this conversation as resolved.
Show resolved Hide resolved

_NOTE_: Many modules are dependent of each other. This means that some models will implicitly run other modules. For example: the `analyze_ceph` module specified above, will be run by all the modules. This means, that one does not need to specify it. It was added here only for reasons of clearity.
boekhorstb1 marked this conversation as resolved.
Show resolved Hide resolved
94 changes: 94 additions & 0 deletions docs/guides/deploy-guide/rookify.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
sidebar_label: Deploy Rookify: Migrate to Rook from Ceph Ansible (technical preview)
sidebar_position: 51
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Deploy Rookify: Migrate to Rook from Ceph Ansible (technical preview)

:::warning

Rookify is developed to migrate from Ceph-Ansible to Rook _in place_ and _without downtime_.
Nevertheless, it is **strongly advised** to test Rookify in a controlled environment beforehand, such as the [OSISM testbed](https://github.com/osism/testbed). Additionally, ensure that all precautionary backups are taken, and any other necessary safety measures are in place.

:::

Rookify is now available in the SCS Reference Implementation (OSISM) and can be deployed as shown in the [paragraph below](#using-the-scs-reference-implementation-osism).

The [Rookify GitHub repository](https://github.com/SovereignCloudStack/rookify) includes a README.md that provides a condensed summary of the information covered here.

## Using the SCS Reference Implementation (OSISM)

:::info

Rookify will be available in OSISM and will be deployable usin osism-commands based on ansible configurations.

:::

But is is not available yet ;)

## Manual Installation

### Download or Clone the Repository

Clone or download Rookify from the [repository](https://github.com/SovereignCloudStack/rookify).

:::tip

Checkout the included options of the added `Makefile` by simply typing `make`.

:::

## Install and Run Locally

1. Navigate to the tool directory:

```bash
cd rookify
```

2. To install Rookify locally, pythons virtualenv will be used (Note: This will install pre-commit in your local user context):

```
make setup
```

This command also checks if you have the required python library for `radoslib` installed. Make sure to have it installed on your linux distribution.

:::tip

Before running rookify, first check all options by using `rookify --help`

:::

To run rookify you can either run it directly from within pythons virtualenv or with help of the make file:

```bash
# directly
./.venv/bin/rookify --help
# using make
make run-local-rookify
```

## Install and Run from within a Container
boekhorstb1 marked this conversation as resolved.
Show resolved Hide resolved

1. Navigate to the tool directory:

2. To install Rookify into a container, podman or docker can be used (Note: in both cases pythons library for `radoslib` needs to be installed locally):

```
make check-radoslib
make up
```

This command uses `docker compose`, so make sure you have that installed as well.

To run rookify you can either enter the container and run rookify from there or use `make run-rookify`.

:::note

Before running rookify, it can be usefull to check all options by using `rookify --help`

:::
17 changes: 17 additions & 0 deletions docs/guides/deploy-guide/services/rookify.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
sidebar_label: Migrate Ceph-Ansible via Rookify (technical preview)
sidebar_position: 51
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Migrate Ceph-Ansible via Rookify (technical preview)

:::warning

This is a technical preview and not recommended for production use yet.

:::

Rookify is now also available in OSISM.
83 changes: 83 additions & 0 deletions docs/guides/operations-guide/rookify.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
sidebar_label: Rookify Operation: Migrate to Rook from Ceph Ansible (technical preview)
---

# Use Rookify: Migrate to Rook from Ceph Ansible (technical preview)

:::warning

Rookify is developed to migrate from Ceph-Ansible to Rook _in place_ and _without downtime_.
Nevertheless, it is **strongly advised** to test Rookify in a controlled environment beforehand, such as the [OSISM testbed](https://github.com/osism/testbed). Additionally, ensure that all precautionary backups are taken, and any other necessary safety measures are in place.

:::

The [Rookify GitHub repository](https://github.com/SovereignCloudStack/rookify) includes a README.md that provides a condensed summary of the information covered here.

## Consider using a pickle file

In order to have a trackable state of process you can add a pickle file. Specify this option in the `config.yaml`:

```yaml title="config.example.yaml"
general:
machine_pickle_file: data.pickle
```

Now you will be able to to view the state of progress by running `rookify --show-state`.
boekhorstb1 marked this conversation as resolved.
Show resolved Hide resolved

:::warning
Rookify will take data.pickle as a source of truth for its operations. If you want to start a clean migration, be sure to delete the file.
:::

## Rookify CLI

### Run

:::warning
Currenlty rookify executes per default
boekhorstb1 marked this conversation as resolved.
Show resolved Hide resolved
:::

Currenlty rookify executes per default. This means: if you run rookify like so `.venv/bin/rookify`, it will start the migration as configured in `config.yaml`.

### --dry-run

:::tip
Run preflight-mode to check if Rookify can connect to your target systems
:::

Rookify has a `preflight-mode` to check if basic commands and the connection to the target systems are working.
You can always run `--dry-run` mode, without any danger that migration processes will be executed.

### --help
boekhorstb1 marked this conversation as resolved.
Show resolved Hide resolved

Run `--help` to see the various options the CLI provides.

### --show
boekhorstb1 marked this conversation as resolved.
Show resolved Hide resolved

Run `--show` if you used a pickel-file (see the configuration-guide concerning the general section). This will show the status of your migration process.

## Debugging and Testing

If you suspect that something is not working in Rookify itself, you can start setting logging to `DEBUG`.

If you suspect that something is not working in Rookify itself, you could start by running the tests. For this you need to have access to rookifies code.
boekhorstb1 marked this conversation as resolved.
Show resolved Hide resolved

### Set logging to debug level

Edit the `config.yaml` and set level to "DEBUG":

```yaml title="config.example.yaml"
logging:
level: DEBUG
format:
time: "%Y-%m-%d %H:%M.%S" # other example: "iso"
renderer: console # or: json
```

You can also set the other formatting options, as indicated by the comments, for further options take a look a the documentation of [structlog](https://www.structlog.org/en/stable/standard-library.html).

### Run tests

Make sure you can reach Rookifies code on you system. Then you can either:
boekhorstb1 marked this conversation as resolved.
Show resolved Hide resolved

1. run `make run-tests-locally` from the working directory of the rookify repository. If you prefer to use a containerized setup, use `make run-tests`.
2. run `.venv/bin/python -m pytest` from the virtual environment of you installation