Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/modules/virtualisation: additional configuration options #349537

Merged
merged 1 commit into from
Dec 5, 2024

Conversation

Yethal
Copy link
Contributor

@Yethal Yethal commented Oct 18, 2024

Added six new configuration options to containerOptions

  • pull Pull policy for the container
  • capAdd Capabilities to add to the container
  • capDrop Capabilities to drop from the container
  • devices Devices to pass through to the container
  • networks Networks container should be connected to
  • privileged Whether container should run in privileged mode

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Oct 18, 2024
@Yethal
Copy link
Contributor Author

Yethal commented Oct 18, 2024

@ofborg test oci-containers

@NixOSInfra NixOSInfra added the 12. first-time contribution This PR is the author's first one; please be gentle! label Oct 18, 2024
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Oct 18, 2024
@Yethal
Copy link
Contributor Author

Yethal commented Oct 18, 2024

@roberth

Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition. I haven't reviewed this module before so I would appreciate if someone could take over, but I've had a look.
It would help me to have tests for this. In particular the hardcoding of networks defaults doesn't seem particularly robust to me.

Comment on lines 328 to 322
capAdd = mkOption {
type = with types; listOf (enum capabilities);
default = [];
description = ''
Capabilities to add to container
'';
example = literalExpression ''
[
SYS_ADMIN
]
'';
};

capDrop = mkOption {
type = with types; listOf (enum capabilities);
default = [];
description = ''
Capabilities to drop from container
'';
example = literalExpression ''
[
SYS_ADMIN
]
'';
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A type that's nicer to use instead of these two options would be lazyAttrsOf (nullOr bool). Now that doesn't have built-in checking of the attribute names, but it does allow for better control of overriding, allowing mkDefault true and mkForce false for example.

That's what arion does for example, but it doesn't check the names; I think docker compose checks it later.
You could check the attribute names by adding to assertions.

A parameter for checking the names would be a nice addition to attrsWith when that lands.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've extended the existing test to cover new config options. I see arion doesn't assert capabilities names either perhaps it's not necessary given a nonexistent capability will cause the service to fail to start

@Yethal Yethal requested a review from roberth October 19, 2024 07:35
@Yethal
Copy link
Contributor Author

Yethal commented Oct 19, 2024

@ofborg test oci-containers

@Yethal
Copy link
Contributor Author

Yethal commented Oct 19, 2024

@benley @mkaito

@Yethal
Copy link
Contributor Author

Yethal commented Oct 19, 2024

@ofborg test oci-containers

1 similar comment
@Yethal
Copy link
Contributor Author

Yethal commented Oct 19, 2024

@ofborg test oci-containers

@Yethal
Copy link
Contributor Author

Yethal commented Nov 12, 2024

@roberth I tried adding other reviewers but it didn't work for some reason, can you add people more suited to review this?

@benley
Copy link
Member

benley commented Nov 12, 2024

Ah sorry I haven't looked at this - I'm about to be offline on a sailboat for a week, but if this is still pending when I get back to land (approximately Nov 21) I will review it then.

@benley benley self-requested a review November 12, 2024 16:49
@Yethal
Copy link
Contributor Author

Yethal commented Nov 25, 2024

@benley Hey, did you manage to get back to land yet?

@Yethal
Copy link
Contributor Author

Yethal commented Dec 3, 2024

@benley

Copy link
Member

@benley benley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, I just have one question before approving. Sorry for the great delay!

nixos/modules/virtualisation/oci-containers.nix Outdated Show resolved Hide resolved
@Yethal Yethal force-pushed the more-configuration-options-for-podman branch from 4326376 to 5a33c10 Compare December 5, 2024 00:20
@benley benley merged commit 04bf3d8 into NixOS:master Dec 5, 2024
19 of 20 checks passed
@Yethal Yethal deleted the more-configuration-options-for-podman branch December 6, 2024 08:12
Comment on lines +298 to +322
capAdd = mkOption {
type = with types; lazyAttrsOf (nullOr bool);
default = { };
description = ''
Capabilities to add to container
'';
example = literalExpression ''
{
SYS_ADMIN = true;
{
'';
};

capDrop = mkOption {
type = with types; lazyAttrsOf (nullOr bool);
default = { };
description = ''
Capabilities to drop from container
'';
example = literalExpression ''
{
SYS_ADMIN = true;
{
'';
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't these be merged into a single capabilities option?

  • true -> add
  • false -> remove
  • null or omitted -> default

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's technically possible, but in my humble opinion it would be a more confusing interface. Part of this module's appeal is that it is a fairly thin abstraction over the underlying container runner implementations; podman and docker (and their various documentation and examples) have separate --cap-add and --cap-drop options so it seems reasonable for us to do the same.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Various declarative wrappers over Docker/podman (Ansible, docker-compose etc) also use separate cap_add and cap_drop options

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A practical benefit of modeling it that way is that you can override the capabilities using the module system.

It is also conceptually simpler, because you don't have to make users bend to the quirks of an listOf str-typed CLI. That's just not a good data model. I shouldn't be able to express this, because it has no clear meaning:

capAdd.SYS_ADMIN = false;
capDrop.SYS_ADMIN = false;

Various declarative wrappers over Docker/podman (Ansible, docker-compose etc) also use separate cap_add and cap_drop options

Arion - a declarative docker-compose wrapper - does not, and I've given that as an example.
I haven't heard of another wrapper; would like to know more.

Apologies if I come across a bit sour. GitHub marked my earlier comment as outdated, so I'll blame them for cutting the conversation short :)
I'd be happy to propose a refactor.

Copy link
Contributor Author

@Yethal Yethal Dec 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I shouldn't be able to express this, because it has no clear meaning:

I agree in principle that this makes no sense but I feel like this is more of a philosophical question, do we want to have the most correct interface or the one that's going to feel the most familiar for users coming in from other ecosystems. Is there a process for asking the community to vote on something (other than starting a thread on Discourse)? If the users decide they prefer the more correct approach vs the more compatible one I'd be happy to rewrite this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have a defined process for this, and I believe that's a good thing.
Despite occasionally being the right thing to ask, "we need more data" is also a really good way to kill an effort.
Also data is not enough to drive development. We also need "human" factors like reasoning and creative thinking to make that work.

I think the crux of the disagreement here is whether or not we should "improve" the things when we wrap or integrate them.
This isn't unique to these flags. It happens all the time. At a large scale we resolve this tension by having both Nixpkgs and NixOS, but even within NixOS, we have multiple layers of abstraction, defined not so much by software component boundaries, but by a causal relationship between options. For instance, we have services.something options, which affects systemd options, which affect environment.etc. This is good, because we can't make assumptions about the level of abstraction users need to work with, and each "layer" can generally be worked with simultaneously without creating conflict.

A similar pattern is possible with RFC-42 style settings, where a service option can effect a high level, perhaps NixOS-specific behavior by contributing to the settings option.
In this situation, the settings correspond directly to an existing configuration format, so they cover the need for the config to be easy to manipulate for existing users of that service.
In this case, the extraOptions cover that need.

RFC 42 provides a recommendation about option quantity

Quality is encouraged over quantity: Authors should spend more time on writing documentation, NixOS tests or useful high-level abstractions.

By adding support for overriding, we make these options/option more useful, and we abstract away the ambiguous and incorrect inputs.

(Also we probably have a bug if the backend doesn't throw an error when a capability is both added and removed. Haven't checked.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like this?
#363574

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You know what, after reading #363574 I think I actually withdraw my objection. That interface and option description make it quite understandable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 12. first-time contribution This PR is the author's first one; please be gentle!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants