Skip to content

Commit

Permalink
tests/storage-volumes-vm: Root volume disk device attachments (#366)
Browse files Browse the repository at this point in the history
This should have a check for all corner cases around VM root volume
attachments (coverage for canonical/lxd#14532):
- security.protection.start allows one other VM to attach the machine's
root disk, and can only be removed if the disk is not attached
- security.shared allows unchecked attachments of root disks
- VM attachments are correctly reported in used_by
- hotplug of VM root attachments works (as this is the method
reccomended by the docs to avoid UUID/LABEL conflicts)

I ran into an interesting behavior of udev in Noble VMs while testing
that slowed this down a bit. In order for udev to create a symlink for
`SCSI_IDENT_SERIAL`
(`/dev/disk/by-id/scsi-SQEMU_QEMU_HARDDISK_lxd_virtual--machine-vm5`), I
had to hotplug the device in an `ubuntu:24.04` VM. Starting the VM from
cold didn't create the link, and `ubuntu-minimal:24.04` VMs never
created it at all.

Without hotplug or in an `ubuntu-minimal:24.04`:
```
# udevadm info /dev/sdb
...
S: disk/by-id/scsi-0QEMU_QEMU_HARDDISK_lxd_virtual--machine
S: disk/by-path/pci-0000:02:00.0-scsi-0:0:1:1
S: disk/by-diskseq/10
E: DEVPATH=/devices/pci0000:00/0000:00:01.1/0000:02:00.0/virtio6/host0/target0:0:1/0:0:1:1/block/sdb
E: DEVNAME=/dev/sdb
...
E: ID_SERIAL=0QEMU_QEMU_HARDDISK_lxd_virtual--machine
E: ID_SERIAL_SHORT=lxd_virtual--machine
E: ID_SCSI_SERIAL=lxd_virtual--machine-v2
E: ID_BUS=scsi
...
E: DEVLINKS=/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_lxd_virtual--machine /dev/disk/by-path/pci-0000:02:00.0-scsi-0:0:1:1 /dev/disk/by-diskseq/10
...
```
With hotplug:
```
# udevadm info /dev/sdb
...
S: disk/by-id/scsi-SQEMU_QEMU_HARDDISK_lxd_virtual--machine-vm5
S: disk/by-path/pci-0000:02:00.0-scsi-0:0:1:1
S: disk/by-id/scsi-0QEMU_QEMU_HARDDISK_lxd_virtual--machine
S: disk/by-diskseq/15
...
E: DEVPATH=/devices/pci0000:00/0000:00:01.1/0000:02:00.0/virtio6/host0/target0:0:1/0:0:1:1/block/sdb
E: DEVNAME=/dev/sdb
...
E: SCSI_IDENT_SERIAL=lxd_virtual--machine-vm5
E: SCSI_IDENT_LUN_VENDOR=lxd_virtual--machine
...
E: ID_SERIAL=0QEMU_QEMU_HARDDISK_lxd_virtual--machine
E: ID_SERIAL_SHORT=lxd_virtual--machine
E: ID_SCSI_SERIAL=lxd_virtual--machine-vm5
...
E: DEVLINKS=/dev/disk/by-id/scsi-SQEMU_QEMU_HARDDISK_lxd_virtual--machine-vm5 /dev/disk/by-path/pci-0000:02:00.0-scsi-0:0:1:1 /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_lxd_virtual--machin>
...
```

I worked around it by using a shorter LXD device name so that the
symlink name wouldn't be truncated; not sure if this is worth chasing
further.
  • Loading branch information
tomponline authored Jan 22, 2025
2 parents 7f3b918 + 6abe711 commit b4a1e6e
Showing 1 changed file with 80 additions and 0 deletions.
80 changes: 80 additions & 0 deletions tests/storage-volumes-vm
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,86 @@ do
lxc storage volume detach "${poolName}" vol3 v1
lxc storage volume detach "${poolName}" vol6 v1 || true # optional ISO

# attach VM root volumes
if hasNeededAPIExtension instance_root_volume_attachment; then
empty_vm_size=8KiB
if [ "${poolDriver}" = "powerflex" ]; then
empty_vm_size=8GiB
fi

lxc init --empty --vm v2 --storage "${poolName}" --device root,size="${empty_vm_size}"
lxc init --empty --vm v3 --storage "${poolName}" --device root,size="${empty_vm_size}"

# Requires either security.shared or security.protection.start
! lxc storage volume attach "${poolName}" virtual-machine/v2 v1 || false

lxc config set v2 security.protection.start=true

# security.protection.start on a VM allows exactly one other attachment
lxc storage volume attach "${poolName}" virtual-machine/v2 v1
! lxc storage volume attach "${poolName}" virtual-machine/v2 v3 || false

# Deleting the instance will fail while it's root volume is in use
! lxc delete v2 || false

# Make sure used_by is calculated correctly
lxc storage volume show "${poolName}" virtual-machine/v2 | grep -qF '/1.0/instances/v1'

# Can't unset security.protection.start when v2's root volume is attached to vm1
! lxc config unset v2 security.protection.start || false

lxc storage volume detach "${poolName}" virtual-machine/v2 v1

# Unset security.protection.start works when not attached
lxc config unset v2 security.protection.start

lxc config set v2 security.protection.start=true
lxc storage volume attach "${poolName}" virtual-machine/v2 v1

lxc storage volume set "${poolName}" virtual-machine/v2 security.shared=true

# Unset security.shared works when security.protection.start=true
lxc storage volume unset "${poolName}" virtual-machine/v2 security.shared
lxc storage volume set "${poolName}" virtual-machine/v2 security.shared=true

# security.shared allows many attachments
lxc storage volume attach "${poolName}" virtual-machine/v2 v3

lxc storage volume show "${poolName}" virtual-machine/v2 | grep -qF '/1.0/instances/v1'
lxc storage volume show "${poolName}" virtual-machine/v2 | grep -qF '/1.0/instances/v3'

lxc config unset v2 security.protection.start

# Detach so that we can double-check hotplug
lxc storage volume detach "${poolName}" virtual-machine/v2 v1

# Make sure that the devices actually show up and can be mounted
lxc start v1
waitInstanceReady v1

# Specify a different device name here; udev appears to be truncating the
# default name when it creates the /dev/disk/by-id/scsi* symlinks, so use
# a shorter name to prevent truncation.
lxc storage volume attach "${poolName}" virtual-machine/v2 v1 v2-root
lxc exec v1 -- test -L /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_lxd_v2--root
lxc stop --force v1

# Can't unset security.shared when v1's root volume is attached elsewhere
! lxc storage volume unset "${poolName}" virtual-machine/v2 security.shared || false

# Instances cannot be moved when their root disk is attached to another instance
! lxc move v2 v4 || false

lxc storage volume detach "${poolName}" virtual-machine/v2 v1
lxc storage volume detach "${poolName}" virtual-machine/v2 v3

lxc storage volume unset "${poolName}" virtual-machine/v2 security.shared

lxc delete v2 v3
else
echo "==> Skipping instance root attachment tests, not supported"
fi

echo "==> Deleting VM"
lxc delete v1

Expand Down

0 comments on commit b4a1e6e

Please sign in to comment.