New User and Group handling

[ffesti]

Currently the creation of users and groups are left to the packagers and is though done differently between different distribution and as %pre scriptlets. This has several drawbacks:

• It is not known which packages create which users
• Packages may ship files with users/groups that don't exist yet
• User/group creation cannot be used as dependencies or for ordering
• It is not possible to check if users/groups get created properly
• It is hard for tools generating images and trying to not (want to) run scriptlets.

Because of this we are looking for a more declarative way of handling users and groups. They are part of the core business of rpm (putting files on disk) but are currently treated as an secondary, accidental detail of the OS. (Initial ticket without much info: #1032)

Current implementations

Inside RPM

Setting user and group of files

• fchown(at) in lib/fsm.c
• rpmugUid() / rpmugGid() in lib/rpmug.c via getpwnam(3)/getgrnam(3)
• Not handling chrooting properly as glibc can't re-load the users/groups

In packages

• %pre scripts execute useradd
• shipping pre-loaded /etc/passwd and friends

Distributions

We collect the implementation details and requirements from different distributions here, to be able to come up with a solution that will
work for everybody. Please add the details of your distribution in the discussion and I will add it to this document.

Fedora

• setup package ships /etc/passwd and friends with 13 system accounts
• Over 500 scriptlets with useradd
• Over 400 scriptlets with groupadd
• %sysusers_create_compat turns sysusers.d files into scriptlets
  • tool chain part of systemd (systemd-rpm-macros)
  • See /usr/lib/rpm/sysusers.generate-pre.sh
  • Does not actually use systemd-sysusers but useradd/groupadd
  • only ~50 packages using it

OpenSUSE

• Special packages containing system users
• Provides and Requires for users and groups
  • Provides: user()
  • Provides: group()
  • Requires(pre): user()
  • Requires(pre): group()
  • https://build.opensuse.org/package/view_file/openSUSE:Factory/sysuser-tools/sysusers.prov
  • https://build.opensuse.org/package/view_file/openSUSE:Factory/sysuser-tools/sysusers.attr
• sysusers.d -> scriptlets solution with macros (similar to Fedora)
  • https://en.opensuse.org/openSUSE:Packaging_guidelines#Users_and_Groups

Your Distribution

• tbd

Challenges and Requirements

• Packages need to be able to provide user/group for their own files
  • need to be created around %pre
  • Last posibility just before setting file meta data
• Empty systems don't have /etc/passwd nor useradd
  • Could use tools from the outside system useradd --root
  • Could rely on package with /etc/passwd being installed first
• There is systemd-sysusers
  • but not everyone uses systemd
  • There is a time before systemd is installed
  • It works with files in /etc/sysusers.d and /usr/lib/sysusers.d
  • We need the new users/groups before setting meta data on files
• UIDs/GUIs and what to do if user/group already exists with a
  different one
• There can be only one of each scriptlets. So autogenerating
  scriptlets is too messy
• Needs to co-exist with current solutions - for quite a while

Solutions

There is currently no final decision on how to implement this feature. So here are some parts and pieces:

• Borrow syntax and feature set from systemd-sysusers files
  • Declaration in the spec file
  • Point to sysusers.d file as an alternative
• Have configurable useradd command that may change during
  installation of the system
• Read and Write /etc files ourselves
• Have separate USERS and GROUPS tags, e.g.

User(httpd): 404 "HTTP user"
User(postgres): - "Postgres db" /var/lib/pgsql /usr/libexec/postgresdb
Group(input): - - 

• %sysusers section that resembels sysusers.d files
  • could use -f to read from sources or build root

%sysusers
u     httpd    404   "HTTP User"
u     postgres -     "Postgresql db" /var/lib/pgsql /usr/libexec/postgresdb
g     input    - 

• Autogenerate Provides: user(foo) and Provides: group(foo)
  • Generate Requires from file ownership is more tricky
    • User / groups are not avaialble to dep generators
    • Breaks if not all users/groups create Provides
    • Could use OrderByRequires as stepping stone

We basically need 3 separate pieces:

• Spec syntax
• Header data
• Install time implementation
  • Using useradd and groupadd looks like a viable option
    • Can be used from outside the chroot
    • May need a generic config
    • Need creating the /etc files empty on first call
    • Use usual script and macro pointing to it approach to allow distributions or OSs tune to their needs.

Solution

After prototyping, looking at the bits and pieces that are already in use and discussions we are currently at the following solution:

• Use Provides: to store user and group data.
• Format is user($USERNAME) = base64encoded systemd-sysusers line or group($GROUPNAME) = base64encoded systemd-sysusers line
• rpm checks for those before installing the package and hands the decoded line to a helper script that will invoke systemd-sysusers (this may be replaced with a script using useradd/groupadd in distributions not shipping systemd
• Use the tools from the system currently running rpm so this will work even on empty chroots
• A dependency generator similar to sysuser-tools will create those provides automatically form sysuser.d files
• A macro allows creating these provides more easily in the spec file
• An extension tag allows showing these tags conveniently in human readable form
• The usual disablers and macros to control the specifics or alter the implementation will be added

Benefits of this approach is that the changes to rpm are relatively small and only a few pieces needs to be added to the rpm code implementation. It also is similar to what (some) distributions have already be doing and builds on that. Provides with the user and group syntax not matching this pattern will be ignored. So current packages should not change behavior as long as they are not rebuild with the new dependency generator enabled.
Keeping large parts of the implementation in helper scripts and macros keeps it easy for distributions to adjust it too their needs and style of user and group handling.

In a second step we can add Requires: to use the user and group information for ordering.

[ffesti]

I will update the top post as a design document and add your answers and suggestions so they don't get lost in the discussion below. So the first questions to every one is:

How does your distribution handle user and group creation? What features would you hope for and what issues solved? I am especially interested in what do people in more exotic places (like other OSs than Linux) do.

[pmatilai]

* Autogenerate Provides: user(foo) and Provides: group(foo)
  
  * Generate Requires from file ownership is more tricky        
    * User / groups are not avaialble to dep generators
    * Breaks if not all users/groups create Provides

We can quite easily generate the user/group provide/requires from the C-side, it just needs to have a disabler. Or, maybe this could be an excuse to make the relevant info available to Lua which could then do this from a parametric generator.

Breaking is a feature I think: this will simply force packages adding users or groups from their scriptlets to add manual provides for them, which will make things easier to track on a distro-level.

[lnussel]

In openSUSE the user() and group() provides are generated automatically if a package has systemd sysuser files: https://build.opensuse.org/package/view_file/openSUSE:Factory/sysuser-tools/sysusers.prov https://build.opensuse.org/package/view_file/openSUSE:Factory/sysuser-tools/sysusers.attr

FWIW mentioning this here for completeness, users and groups do not necessarily need to be created in %pre stage in all cases. For transactional systems that only update the os in /usr and apply any potential data and config migrations at boot, users would also get created at boot only.

[ffesti]

Quick note: "m" lines in the sysusers.d files could be used to create Requires to the user and group.

[ffesti]

See #2432

[pmatilai]

User/group handling was implemented in #2432, documentation at https://rpm-software-management.github.io/rpm/manual/users_and_groups.html

Closing this discussion to avoid folks getting confused over all the speculated variants in here, lets open a new discussion (and/or tickets of course) around what was implemented if necessary.