How to properly execute Sanoid concurrently with different configs?

[tschoening81]

I have Sanoid installed to create snapshots of internal storage and want to use it for external USB backup as well. I have a backup script where I want to call Sanoid manually with a special config file using --configdir=/etc/sansoid/bak_usb. What I'm wondering about is the other options for cache and lock files. Both instances of Sanoid work on different and exclusive ZFS pools.

So, is it required to specific individual cache-dir and run-dir as well?

I guess if I don't do so, the Sanoid instance handling system internal storage and that for backup only simply can't run concurrently? So whenever the execution time overlaps, I run into some error?

Does the cache-dir needs to be maintained at the USB backup disk?

I'm having trouble understanding what the cache file is actually used for. My USB backup gets switched monthly. So what happens to the cache file if it's stored at the system and outdated with respect to a new USB disk used? Is it necessary to put the cache onto the USB disk? I don't care about performance too much, so if Sanoid would take a minute more to do it's stuff, that would be OK for me. Therefore I currently prefer keeping things next to each other, which means the cache would be something like /var/cache/sanoid/bak_usb instead of /media/bak_usb/sanoid/cache.

Thanks for your answers!

[Pajkastare]

The way the current code is written, the snapshot cache file is used to avoid getting all snapshots from all datasets in all zpools every time sanoid needs to do something - Which is a good thing, because ZFS is slow to do this, and ZFS by itself does not seem to cache this value at all. (zfs get -Hrpt snapshot creation, currently here, took me about 7.5 seconds, consistently, for two zpools on NVMe and SSD drives, respectively, with a total of roughly 11000 snapshots.)

Also the way the code is written, the sanoid snapshot cache file includes all snapshots for all zpools. So you should definitely keep the same cache file for all sanoid instances, in order to reduce the time spent waiting for snapshot info.

Your question is not clear regarding exactly what you put on your USB drive, but if you want to keep the zpools separate from each other, perhaps you should handle your USB drive (with sanoid) from within a VM, and pass through the USB drive to the VM directly.

[tschoening81]

Your question is not clear regarding exactly what you put on your USB drive

One ZPOOL containing mutliple datasets and their snapshots containing RSYNC-maintained backup files.

but if you want to keep the zpools separate from each other, perhaps you should handle your USB drive (with sanoid) from
within a VM, and pass through the USB drive to the VM directly.

I don't get how that targets my question... :-) There's no VM involved and I already have multiple different ZPOOLs, e.g. one for the root file system, one for slow HDDs and one or better multiple for backup purposes on different USB-disks.

The question is how the cache works if I switch USB disks, which is actually switching ZPOOLs on a monthly basis, so that different snapshots are recognized when Sanoid works on new USB disks at least for the first time. I didn't encounter any errors in the last few months, so I guess it just works, the cache is oiutdated and updated at some point. As long as the same ZPOOL is connected using USB, the cache helps, if a USB diosk is switched it gets outdated and updated and might help afterwards again.