How to handle dynamic toplevel directories that may be needed?

[cgwalters]

Today "podman machine" is a VM that wants to bind mount (via virtiofs) paths into the VM that match the host which could be platform specific or even dynamic. On MacOS for example, the home directory is called /Users and the way things work is that the container bind mounts (in the VM) that same absolute path.

How to deal with this with a readonly /?

[cgwalters]

In the short term my recommendation is for cases like podman machine to pre-create in their container build all the toplevel directories they may need by default. It's OK to ship an empty /Users even in a VM which may run on Linux.

That said, for those that have a need for truly dynamic management, a simple option is to enable transient root. However that's a big hammer.

What would be possible to do is enable transient root, but add code into the initramfs which creates those toplevel directories and then remounts the root readonly. This would mean discovery of the desired state would need to be handled in the initramfs.

Finally, one option we could add is something like root = transient-ro where we'd allocate the overlayfs upper, but still keep it read-only by default. This would make it easy for code running in the real root to unshare the mount namespace, mount it writable and mutate it while still keeping it read-only for most use cases. This would be a pretty easy addition to ostree-prepare-root.

[praveenkumar]

It's OK to ship an empty /Users even in a VM which may run on Linux.

you mean this /User should be symlink to /var/User because then only sub directories can be created part of mount. Because just creating an empty top level directory also become immutable as per https://containers.github.io/bootc/filesystem.html#other-toplevel-directories right?

[cgwalters]

No, it's possible to create a mountpoint on top of a read-only directory. So my recommendation is that /Users is part of the container image always, and should not be a symlink to /var.

Although I guess if in this use case we don't want to mount all of /Users but we only want /Users/walters or whatever, then indeed it'd need to be a symlink into /var and the subdirectories created dynamically as you say.

[praveenkumar]

So my recommendation is that /Users is part of the container image always, and should not be a symlink to /var.

I need to experiment with that.

Although I guess if in this use case we don't want to mount all of /Users but we only want /Users/walters or whatever, then indeed it'd need to be a symlink into /var and the subdirectories created dynamically as you say.

So in case of openshift-local (not sure about podman-machine) we mount $HOME which means /Users/<respective_user> so we do need to do symlink to /var.

Edit: Checked with podman-machine and this also mount $HOME