Resize NVMe sector size on existing P4510 disks in eddie

[Firefishy]

The P4510 NVMe disks in /dev/md1 array in eddie use 512 byte sectors instead of 4096 byte sectors. The disks should be removed from the array, sector size changed (format) and disks re-added to the array

disk    INTEL SSDPE2KX020T8__1                     /dev/nvme8n1
disk    INTEL SSDPE2KX020T8__1                     /dev/nvme9n1

The LBA format reports 4096 bytes is the best performance:

LBA Format  0 : Metadata Size: 0   bytes - Data Size: 512 bytes - Relative Performance: 0x2 Good (in use)
LBA Format  1 : Metadata Size: 0   bytes - Data Size: 4096 bytes - Relative Performance: 0 Best

This can be done as part of the upgrade in #1117 to reduce any data corruption risks (ensure full parity):

Likely steps:

# Insert new NVMe disks from #1117 
sudo mdadm /dev/md1 --add /dev/nvme10n1 --replace /dev/nvme8n1 --with /dev/nvme10n1
sudo mdadm /dev/md1 --add /dev/nvme11n1 --replace /dev/nvme9n1 --with /dev/nvme11n1
sudo mdadm /dev/md1 --remove /dev/nvme8n1
sudo mdadm /dev/md1 --remove /dev/nvme9n1
cat /proc/mdstat (ensure nvme8n1 and nvme9n1 are removed)
sudo nvme format --lbaf=1 /dev/nvme8n1
sudo nvme format --lbaf=1 /dev/nvme9n1
sudo mdadm /dev/md1 --add /dev/nvme8n1 /dev/nvme9n1
sudo mdadm --grow /dev/md1 --raid-devices=12
sudo resize2fs /dev/md1

[Firefishy]

Link #1119 for karm twin.

[Firefishy]

All but final 2 steps done. md array now growing from 10 -> 12 disks.

[Firefishy]

The resize2fs step errors out with: resize2fs: New size too large to be expressed in 32 bits

The fix is documented here, but needs the filesystem unmounted.

[Firefishy]

The block size has been updated and filesystem has been resized. All done.

Enabling 64bit flag on filesystem.

umount
e2fsck -fD -C 0 /dev/md1 
resize2fs -b /dev/md1
resize2fs /dev/md1
e2fsck -fD -C 0 /dev/md1 
tune2fs -O metadata_csum /dev/md1
e2fsck -fD -C 0 /dev/md1 
mount