runAsUser: 1000 in securityContext causes error in Openshift

[MarkusFlorian79]

Openshift does dynamically attribute defined user ranges. Therefore the user 1000 is in general not a valid user.
In version 1.6 it was removed already (### Removal of default security context UID setting) , but was reintroduced with #909.
The securityContext can be defined without the runAsUser and everything works fine.

[rainest]

Are you having issues clearing that default? Although Helm's default value system is cumbersome to work with, an explicit null does appear to clear this:

$ helm template ana kong/kong --set containerSecurityContext.runAsUser=null | grep -iA9 securitycontext | head -11
        securityContext:
        
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          seccompProfile:
            type: RuntimeDefault
--

---

As #909 states, this doesn't have anything to do the reason mentioned in #1003

Versions of Kong prior to 2.0 and Kong Enterprise prior to 1.3 use Docker images that required setting a UID via Kubernetes in some environments

This is for compliance with Kubernetes 1.23+ default Restricted security policy (under "Running as Non-root user") to not run as root. Although docs state that nil should be allowed, the full story is more complex. The code check for this has failure cases for nil UID.

IIRC in practical testing I could not deploy into a restricted-enforced environment without adding the explicit run as user. I'm not familiar with how the username input to that verification function is derived, I expect it's looking at the container metadata and seeing the kong user we create, whose UID is determined by the distro package, not the container image build.

[MarkusFlorian79]

We are using the chart as subchart. Setting the runAsUser=null results in the default value 1000 being used.

[rainest]

Testing confirms we do indeed hit that restriction if we remove this. To demonstrate, I created a restricted namespace and installed several configurations into it and default (which has no policy configured):

48853  kubectl create -f https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/security/podsecurity-restricted.yaml
48855  helm install unrestricted-null kong/kong --set containerSecurityContext.runAsUser=null
48856  helm install restricted-default -n my-restricted-namespace kong/kong
48857  helm install restricted-null -n my-restricted-namespace kong/kong --set containerSecurityContext.runAsUser=null
48859  helm install restricted-null-asroot -n my-restricted-namespace kong/kong --set containerSecurityContext.runAsUser=null --set containerSecurityContext.runAsNonRoot=false

Removing it actually breaks even the unrestricted install. The check actually confirms whether the container complies with its runAsNonRoot setting regardless of whether the namespace policy requires it:

11:13:05-0800 esenin $ kubectl get deploy
NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
httpbin                  1/1     1            1           36m
unrestricted-null-kong   0/1     1            0           20m

11:13:12-0800 esenin $ kubectl describe po unrestricted-null-kong-7c464b8c6f-57kzj| grep -i root
    ConfigMapName:           kube-root-ca.crt
  Warning  Failed     19m (x12 over 21m)  kubelet            Error: container has runAsNonRoot and image has non-numeric user (kong), cannot verify user is non-root (pod: "unrestricted-null-kong-7c464b8c6f-57kzj_default(dabf4849-283d-4956-8bb1-60f9ad00d09a)", container: clear-stale-pid)

In the restricted namespace you'll get the same failure if you install with runAsNonRoot=true and no UID. You'll additionally get a failure on the ReplicasSet if you toggle runAsNonRoot off because it violates policy. Though not shown, installing with --set containerSecurityContext.runAsUser=null --set containerSecurityContext.runAsNonRoot=false in default does work:

11:13:28-0800 esenin $ kubectl get deploy -n my-restricted-namespace 
NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
restricted-default-kong       1/1     1            1           20m
restricted-null-asroot-kong   0/1     0            0           17m
restricted-null-kong          0/1     1            0           20m

11:13:44-0800 esenin $ kubectl describe po -n my-restricted-namespace restricted-null-kong-65ddbd6947-tpxdq| grep -i root
    ConfigMapName:           kube-root-ca.crt
  Warning  Failed     18m (x12 over 20m)  kubelet            Error: container has runAsNonRoot and image has non-numeric user (kong), cannot verify user is non-root (pod: "restricted-null-kong-65ddbd6947-tpxdq_my-restricted-namespace(3dc95a4c-cd06-45f0-8269-fa25bdd2507b)", container: clear-stale-pid)

11:14:41-0800 esenin $ kubectl describe replicasets.apps -n my-restricted-namespace restricted-null-asroot-kong | grep -i forbidden | head -1 
  Warning  FailedCreate  18m                  replicaset-controller  Error creating: pods "restricted-null-asroot-kong-7cf76d44d7-88r8z" is forbidden: violates PodSecurity "restricted:latest": runAsNonRoot != true (containers "clear-stale-pid", "ingress-controller", "proxy" must not set securityContext.runAsNonRoot=false)

So AFAICT nulling the setting is the only option if we don't want to remove restricted compliance and further indicate that the container may run as root.

The implementation of the check suggests that there may be a way to create a container with no user that then passes the check, but I suspect that's not actually possible and is just an entirely hypothetical situation the code can handle. I've asked SIG Auth if they know otherwise, but in any case I think it's likely we'd clear the user from the container image build.

[rainest]

We are using the chart as subchart. Setting the runAsUser=null results in the default value 1000 being used.

Fascinating. The subchart implementation never fails to amaze: helm/helm#12637

It looks like the Pod enforcement can indeed permit runAsNonRoot=true without a UID if you use a numeric ID at build time, so patching one of the standard Dockerfiles and building gets you something that's compatible:

$ git diff
diff --git a/Dockerfile.deb b/Dockerfile.deb
index b367869..34f65a6 100644
--- a/Dockerfile.deb
+++ b/Dockerfile.deb
@@ -39,7 +39,7 @@ RUN set -ex; \
 
 COPY docker-entrypoint.sh /docker-entrypoint.sh
 
-USER kong
+USER 1000
 
 ENTRYPOINT ["/docker-entrypoint.sh"]

Having built that, in the unrestricted namespace:

$ helm install uuu kong/kong --set containerSecurityContext.runAsUser=null --set image.repository=tmp --set image.tag=tmp --set image.effectiveSemver=3.5.0

$ kubectl get po                                                                                                                                           
NAME                                      READY   STATUS                            RESTARTS   AGE
boo-kong-d7d68dcbb-hbvv5                  2/2     Running                           0          30m
httpbin-66c5b5bc6b-xnjxp                  1/1     Running                           0          74m
unrestricted-null-kong-5bf5b64798-mzpr2   0/2     Init:CreateContainerConfigError   0          11m
uuu-kong-57ffbc5bfb-ssx2v                 2/2     Running   

The controller already uses a numeric USER argument, so it doesn't need a custom image.

It looks like you should also be able to instruct OpenShift that some given range of UIDs is acceptable for a namespace (or project). Is that an option? It's probably simpler than maintaining a custom image, even if it's a simple one.

https://www.ibm.com/docs/en/cloud-paks/cp-management/2.2.x?topic=collection-adding-user-id-group-id-in-dockerfile
https://docs.openshift.com/dedicated/authentication/managing-security-context-constraints.html#security-context-constraints-pre-allocated-values_configuring-internal-oauth
https://developer.ibm.com/learningpaths/secure-context-constraints-openshift/deployment-specify-permissions

Some of that content suggests you can maybe set this permitted range at the Deployment level, but I'm not sure. If that's actually allowed, we could probably add those annotations to the chart resources.

[MarkusFlorian79]

Thx for your efforts. For the time being we will set the runAsUser explicitly, even if it could change after recreation of the project.