Add way to manage ACLs and Quotas for Service Accounts through the User Operator

[scholzj]

With #10317 we add support for service-account-based authentication. We should also add a way to manage Simple ACLs and Quotas with KafkaUser resources and User Operator.

[scholzj]

The tricky part he is how to deal with namespaces 🤔

[scholzj]

Triaged on the Community call on 8.8.2024: This would make sense. But given the complication with namespaces, a proposal should be written first.

[sebypp]

Background and simplified overview for OAuth auth/authz

1. authn: Principal username is extracted from token (defaults to sub claim). More information here.
2. authz: username obtained from [1] is matched against Principals defined via KafkaUser CRD and appropriate authz is applied.

Problem statement

KafkaUser CRD maps the metadata.name to actual Kafka Principal, which imposes restrictions on what usernames can look like. In plain English, a Kubernetes JWT token (example below) would default to its sub.

{
  "aud": [
    "https://kubernetes.default.svc"
  ],
  "exp": 1731613413,
  "iat": 1700077413,
  "iss": "https://kubernetes.default.svc",
  "jti": "ea28ed49-2e11-4280-9ec5-bc3d1d84661a", 
  "kubernetes.io": {
    "namespace": "kube-system",
    "node": {
      "name": "127.0.0.1",
      "uid": "58456cb0-dd00-45ed-b797-5578fdceaced"
    },
    "pod": {
      "name": "coredns-69cbfb9798-jv9gn",
      "uid": "778a530c-b3f4-47c0-9cd5-ab018fb64f33"
    },
    "serviceaccount": {
      "name": "coredns",
      "uid": "a087d5a0-e1dd-43ec-93ac-f13d89cd13af"
    },
    "warnafter": 1700081020
  },
  "nbf": 1700077413,
  "sub": "system:serviceaccount:kube-system:coredns" # This would be the actual Kafka Principal username.
}

The system:serviceaccount:kube-system:coredns is not a valid username that we can use for the KafkaUser CRD resource.

Possible solutions

1.Implement advanced parsing for userNameClaim using JsonPath and/or jackson-jq to something more dynamic than just extracting a claim.
2. Add a custom annotation to the KafkaUser CRD that will be used as Kafka Principal username instead of the metadata.username defined in the CRD.