[WIP] Volume mount paths compatibility on darwin

[tricktron]

CoreOS disallows mounting at /root because it is immutable.

As an alternative, it is recommended to mount at /mnt, e.g: podman machine init -v /Users:/mnt/Users on darwin. This, however, breaks compatibility with other tools, e.g. docker desktop, colima when specifying mounts:

\# docker-compose.yml
...
volumes:
    - ./app:/app

• Works in colima and docker desktop.
• Fails with podman run because it is missing the /mnt prefix.

As a result, I can't use podman and docker desktop interchangeably
with one single docker-compose.yml file.

This WIP tries to solve that with the following convention:

• podman machine: every target mount path into coreOS is internally prefixed with
  /mnt so that it can be mounted.
• podman run: every source mount path is internally prefixed with /mnt if you
  are on darwin.

Now the last sentence if youare on darwin may break a lot stuff, e.g what happens when using the podman remote client on linux with server vs. when using it with a linux vm. Since I am brand new to this project, it may very well be that this approach is heading into the wrong direction. But it at least works on my m1 Mac and may serve as a good starting point for a discussion for v4.1.

There are multiple approaches to solving this issue. I can currently think of 4:

• Removing the immutability of / in the coreOS image. This allows mounting on root. Allow creation of immutable directories below root folder coreos/fedora-coreos-tracker#270
• Workaround the immutability by first disabling it, mounting the volume and then enabling it again. Currently in master:
  

  podman/pkg/machine/qemu/machine.go

  Lines 477 to 488 in 4a242b1

  	// create mountpoint directory if it doesn't exist
  	// because / is immutable, we have to monkey around with permissions
  	// if we dont mount in /home or /mnt
  	args := []string{"-q", "--"}
  	if !strings.HasPrefix(mount.Target, "/home") || !strings.HasPrefix(mount.Target, "/mnt") {
  	args = append(args, "sudo", "chattr", "-i", "/", ";")
  	}
  	args = append(args, "sudo", "mkdir", "-p", mount.Target)
  	if !strings.HasPrefix(mount.Target, "/home") || !strings.HasPrefix(mount.Target, "/mnt") {
  	args = append(args, ";", "sudo", "chattr", "+i", "/", ";")
  	}
  	err = v.SSH(name, machine.SSHOptions{Args: args})

• Mounting under /mnt, then disabling the immutability using the above trick by @baude, then creating a symlink in root, then enabling immutability again.
• This pr: using a convention.
• ...

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: tricktron
To complete the pull request process, please assign mtrmac after the PR has been reviewed.
You can assign the PR to them by writing /assign @mtrmac in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Luap99]

The remote client is generic and not just limited to podman machine. You cannot just change the path for all darwin installs. This will break a lot of things.
If the code from @baude works I see no reason why we would need this.

[tricktron]

@Luap99 I was developing that in parallel to @baude's solution and since it worked well enough I wanted it to share it here. According to the tests, it doesn't even break that much.

I lack the broader background to judge which solution is better.

But in my opinion it doesn't make sense to use an immutable / only to disable it for mounting and then reenabling it again.

But this is up to you and the community to decide, this are just my 2 cents.

[baude]

we are hoping to work with the fcos community to provide a better long-term solution. we kicked around some ideas the other day but never really came to a conclusion. @dustymabe @cgwalters here is a pr that cleans up some of what I did but nevertheless shows we need to figure this out ... i dont think i knew the other day that docker compose defaults to using / !!

[Luap99]

According to the tests, it doesn't even break that much.

Well only because macos/windows and podman machine are completely untested!

[tricktron]

Well only because macos/windows and podman machine are completely untested!

Do you know a way to test this manually?

[tricktron]

we are hoping to work with the fcos community to provide a better long-term solution. we kicked around some ideas the other day but never really came to a conclusion. @dustymabe @cgwalters here is a pr that cleans up some of what I did but nevertheless shows we need to figure this out ... i dont think i knew the other day that docker compose defaults to using / !!

@baude Any updates?

[baude]

We have no updates to provide on that matter. We have a temporary fix in 4.0.2 for the Mac brew client. We have also decided to mount -v $HOME:$HOME as a default in the next release. It can be overriden, changed, and also you can opt out of any mounting.

[tricktron]

@baude Ok, thanks for the info. Should I close this pr or keep it open?

[rhatdan]

I would say close this, then open a PR to add fields to containers/common, for default paths to volume mount into machines

machine_volumes=[ "$HOME:$HOME", "/tmp:/tmp:ro" ]

Supports both environment variables and paths.

[tricktron]

@rhatdan This pr was an alternative solution to allow compatibility with docker-compose files without mounting at / because the immutability of it discourages it.
@baude already fixed that by just disabling the immutability when trying to mount under / then enabling it again. In my opinion, this is a more of a dirty hack so I presented my alternative solution here. Since his workaround is already in master and does not break things and my solution may break things according to @Luap99, I think we just close this here and I won't open a new pr.

[rhatdan]

We can move this to a discussion.