Skip to content

Latest commit

 

History

History
282 lines (213 loc) · 9.17 KB

01-hypervisor.md

File metadata and controls

282 lines (213 loc) · 9.17 KB

Hypervisor

In this section you will prepare the bare metal host in a way, that it will be capable of running virtualized workload. This will include the initial setup of storage and networking.

Environment Variables

For convenience and readability set the following variables. FEDORA_VERSION defines the release of Fedora that should be used for installing the services machine. The fully qualified domain name (FQDN) in the tree hierarchy of the Domain Name System (DNS) should be equal to SUB_DOMAIN.BASE_DOMAIN.

Adjust SUB_DOMAIN and BASE_DOMAIN to your needs if required. Make sure to set this environment variables are set whenever you are working on the lab.

You can set the environment variables automatically by adding them to the personal initialization file ~/.bash_profile that configures the user environment:

[root@okd ~]# echo "export SUB_DOMAIN=okd" >> ~/.bash_profile
[root@okd ~]# echo "export BASE_DOMAIN=example.com" >> ~/.bash_profile
[root@okd ~]# echo "export FEDORA_VERSION=36" >> ~/.bash_profile
[root@okd ~]# echo "export OKD_VERSION=4.11.0-0.okd-2022-10-28-153352" >> ~/.bash_profile
[root@okd ~]# source ~/.bash_profile

Packages

Install the virtualization tools via the command line using the virtualization package group. To view the packages, run:

[root@okd ~]# dnf groupinfo virtualization

Run the following command to install the mandatory and default packages in the virtualization group:

[root@okd ~]# dnf install @virtualization -y

After installation, start the libvirtd service:

[root@okd ~]# systemctl enable libvirtd --now

Verify that the KVM kernel modules are properly loaded:

[root@okd ~]# lsmod | grep kvm

kvm_amd                55563  0
kvm                   419458  1 kvm_amd

If this command lists kvm_intel or kvm_amd, KVM is properly configured.

Now install all additional required packages:

[root@okd ~]# dnf install git virt-install -y

Hostname

It is also a good idea to set the hostname to the FQDN of the hypervisor machine:

[root@okd ~]# hostnamectl set-hostname --static $SUB_DOMAIN.$BASE_DOMAIN

User

Create the user okd and assign any password you like.

[root@okd ~]# useradd okd
[root@okd ~]# passwd okd

On Fedora, it is the wheel group the user has to be added to, as this group has full administrative privileges. libvirt is needed to manage virtual machines and networks. Those tasks usually requires more permissions. Add the okd user to the groups using the following command:

[root@okd ~]# usermod -aG wheel okd
[root@okd ~]# usermod -aG libvirt okd

Then switch to the user okd.

[root@okd ~]# su - okd

Now run the commands to setup the environment variables again.

Repository

Clone this repository to easily access resource definitions on the hypervisor:

[okd@okd ~]$ git clone https://github.com/raballew/okd-the-hard-way.git

Then replace all occurrences of BASE_DOMAIN and SUB_DOMAIN in the sources files, so that the configuration is tailored to your specific environment.

[okd@okd ~]$ grep -rl "{{ BASE_DOMAIN }}" ~/okd-the-hard-way/src/ | xargs sed -i "s/{{ BASE_DOMAIN }}/$BASE_DOMAIN/g"
[okd@okd ~]$ grep -rl "{{ SUB_DOMAIN }}" ~/okd-the-hard-way/src/ | xargs sed -i "s/{{ SUB_DOMAIN }}/$SUB_DOMAIN/g"

Configure libvirt

If not explicitly stated, the virsh binary uses the qemu:///session URI which will not work in our case, as we need to use virtual networks defined in qemu:///system. Defining LIBVIRT_DEFAULT_URI will configure virsh to connect to the URI specified per default. By appending the export of the environment variable to the .bash_profile, personal initialization for the user okd is configured to use qemu:///system per default.

[okd@okd ~]$ echo "export LIBVIRT_DEFAULT_URI=qemu:///system" >> ~/.bash_profile
[okd@okd ~]$ source ~/.bash_profile

Then fix potential permission issues by running libvirt as okd user instead of qemu.

[okd@okd ~]$ sudo sed -i 's/#user = "root"/user = "okd"/g' /etc/libvirt/qemu.conf
[okd@okd ~]$ sudo sed -i 's/#group = "root"/group = "okd"/g' /etc/libvirt/qemu.conf
[okd@okd ~]$ sudo systemctl restart libvirtd

Storage

Libvirt provides storage management on the physical host through storage pools and volumes. A storage pool is a dedicated quantity of storage usually reserved by a dedicated storage administrator. Storage pools are not required for proper operation of VMs but it is a good way to manage storage related and used by VMs.

Storage Pool

Special disk formats such as qcow2, raw, iso, e.g. are supported by the qemu-img program and used while setting up the VMs. The recommended type of pool to manage this files is dir.

Create the storage pool which will be used to serve the VM disk images:

[okd@okd ~]$ mkdir -p ~/images/
[okd@okd ~]$ virsh pool-define ~/okd-the-hard-way/src/01-hypervisor/storage-pool.xml
[okd@okd ~]$ virsh pool-autostart okd
[okd@okd ~]$ virsh pool-start okd

Volumes

Creating an empty disk image for each VM ensures that the content of each VM is stored in a predefined location. This is not a mandatory step, but it helps to simplify things later on and keep track of which storage is consumed by which VM.

Each node of the cluster will get a 128G large disk attached to it, with exception of the services and storage nodes as their demand is slightly higher:

# The services machine needs a larger disk as it will serve all artifacts
[okd@okd ~]$ qemu-img create -f qcow2 ~/images/services.$HOSTNAME.0.qcow2 256G
# Default sized disks for all OKD nodes
[okd@okd ~]$ for node in \
    bootstrap \
    compute-0 compute-1 compute-2 \
    master-0 master-1 master-2 \
    storage-0 storage-1 storage-2 \
    infra-0 infra-1 infra-2 ; \
do \
    qemu-img create -f qcow2 ~/images/$node.$HOSTNAME.0.qcow2 128G ; \
done
# Additional disks for storage nodes
[okd@okd ~]$ for node in \
    storage-0 storage-1 storage-2 ; \
do \
    qemu-img create -f qcow2 ~/images/$node.$HOSTNAME.1.qcow2 256G ; \
done

Fedora ISO

The services machine is the first machine that needs to be setup. All other VMs will be bootstrapped using Preboot eXecution Environment (PXE) procedures. Therefore the services machine is going to host PXE boot services and more. Fedora offers all required packages to do so and will be used as the operating system on the services VM.

Download the Fedora Server ISO file:

[okd@okd ~]$ curl -X GET "https://download.fedoraproject.org/pub/fedora/linux/releases/$FEDORA_VERSION/Server/x86_64/iso/Fedora-Server-dvd-x86_64-$FEDORA_VERSION-1.5.iso" -o ~/images/Fedora-Server-dvd-x86_64-$FEDORA_VERSION-1.5.iso -L

Network

Virtual Network

It is a good practice to move network traffic into a separate virtual network, but even the default network created by libvirt could be used. The network should have no Network Address Translation (NAT) enabled to setup an isolated network and all desired Media Access Control (MAC) and Internet Protocol (IP) addresses need to be defined.

When creating and starting the network virsh will attempt to create a bridge interface.

[okd@okd ~]$ virsh net-define ~/okd-the-hard-way/src/01-hypervisor/network.xml
[okd@okd ~]$ virsh net-autostart okd
[okd@okd ~]$ virsh net-start okd

Services

Kickstart installations offer a way to automate every task in the installation process. Kickstart files provide answers to all questions asked during the installation process. Therefore, if you provide a Kickstart file when the installation begins, the installation will be partially or fully automated. The Kickstart file for the services machine can be found at services.ks.

The services VM will be the only node with direct internet access trough the default libvirt network. Start the installation of the services VM:

[okd@okd ~]$ USER_PASSWORD=$(openssl rand -hex 128)
[okd@okd ~]$ echo "user --name=okd --password=$USER_PASSWORD --plaintext --groups=wheel" >> ~/okd-the-hard-way/src/01-hypervisor/services.ks
[okd@okd ~]$ virt-install \
    --name services.$HOSTNAME \
    --description "services" \
    --os-type Linux \
    --os-variant fedora$FEDORA_VERSION \
    --disk ~/images/services.$HOSTNAME.0.qcow2,bus=scsi,size=256,sparse=yes \
    --controller scsi,model=virtio-scsi \
    --network network=default \
    --network network=okd \
    --location ~/images/Fedora-Server-dvd-x86_64-$FEDORA_VERSION-1.5.iso \
    --initrd-inject=/home/okd/okd-the-hard-way/src/01-hypervisor/services.ks \
    --extra-args "console=ttyS0,115200 inst.ks=file:/services.ks" \
    --ram 8192 \
    --vcpus 2 \
    --cpu host \
    --accelerate \
    --graphics none \
    --boot useserial=on

Once the installation finished, login with username okd and password equal to the value stored in the USER_PASSWORD variable. Exit the session with CTRL+]. The console can be accessed trough virsh at any time:

[okd@okd ~]$ virsh console services.$HOSTNAME

Connected to domain services
Escape character is ^]

Make sure that the services VM starts automatically:

[okd@okd ~]$ virsh autostart services.$HOSTNAME

Next: Services