Can the consul-login-auth-method be fully customizable?

[nabadger]

Question

Currently (referenced in #3280) we run consul-servers on different k8s clusters to consul-clients. We don't run these in a federated setup (so essentially a single data-center setup).

I have a PoC working where I can pre-create k8s auth methods on the consul-servers that are specific to the consul-clients.

Since the clients are on a different cluster, the k8s-auth-method will have a different k8s-host, CA and service-account-jwt from the default.

In some cases such as https://github.com/hashicorp/consul-k8s/blob/main/charts/consul/templates/connect-inject-deployment.yaml#L196 the auth-method can be overridden.

{{- if .Values.connectInject.overrideAuthMethodName }}
  -acl-auth-method="{{ .Values.connectInject.overrideAuthMethodName }}" \
{{- else if .Values.global.acls.manageSystemACLs }}
  -acl-auth-method="{{ template "consul.fullname" . }}-k8s-auth-method" \
{{- end }}

In other cases such as https://github.com/hashicorp/consul-k8s/blob/main/charts/consul/templates/client-daemonset.yaml#L512 it's essentially hard-coded.

value: {{ template "consul.fullname" . }}-k8s-component-auth-method

---

Should this be fully customizable?

I've also had a peek at consul-k8s code and found that in some cases this is customizable and the auth-method can be configured via server-acl-init ( as per https://github.com/hashicorp/consul-k8s/blob/main/control-plane/subcommand/server-acl-init/connect_inject.go#L164 )

However, we still need this in the client-daemonset as far as I can tell.

The second problem here is managing the creation of those auth-methods - it would be preferable to have consul-k8s handle that like it does with service-acl-init (but i think i would need it for consul-clients one too).

Thanks!

[nabadger]

Note, I've tested this locally and it would require:

• Overrides for the auth-method names
• Overrides for service-accounts name (or a way to specify the exact service-account binding)
• Overrides for RBAC (if service-account names can be custom)

I'm making use of the existing server-acl-init Job to configure auth-methods, although consul-k8s-control-plane does generate a lot of resources based on -resource-prefix (ideally this would be more flexible).