Reconciler error" err="providerClient authentication err: Get "https://10.x.x.x:5000": tls: failed to verify certificate: x509: certificate signed by unknown authority"

[rammeena]

Hi,

I am trying to use magnum-cluster-api driver with magnum in kolla-ansible openstack deployment (1 ctl + 1 comp)

I have deployed cluster-api in kind k8s cluster on a seperate ubuntu VM on KVM.

OpenStack APIs are using https certs self signed private CA certificates by Kolla-Ansible for external APIs.

when I try to create Kubernetes cluster in openstack it shows below error in kind cluster capo-controller-manager :

kubectl -n capo-system logs deploy/capo-controller-manager -f

"Reconciler error" err="providerClient authentication err: Get \"https://10.x.x.x:5000/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" controller="openstackmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="OpenStackMachine" OpenStackMachine="magnum-system/kube-2k150-default-worker-6ljfm-nzmjk-8pnw7" namespace="magnum-system" name="kube-2k150-default-worker-6ljfm-nzmjk-8pnw7" reconcileID="fdca9177-8b21-4df7-b28a-2ee6be9517a2"

ubuntu@k8s-kind:~/debugging$ kubectl -n magnum-system get clusters kube-2k150
NAME         CLUSTERCLASS     PHASE          AGE   VERSION
kube-2k150   magnum-v0.21.2   Provisioning   17h   v1.27.4

when I check certificate secrets in kind cluster it show below:

ubuntu@k8s-kind:~$ kubectl get secret -n magnum-system
NAME                      TYPE                      DATA   AGE
kube-2k150-ca             cluster.x-k8s.io/secret   2      15h
kube-2k150-cloud-config   Opaque                    2      15h
kube-2k150-etcd           cluster.x-k8s.io/secret   2      15h
kube-2k150-proxy          cluster.x-k8s.io/secret   2      15h
kube-2k150-sa             cluster.x-k8s.io/secret   2      15h

If I edit the certificate value in secret kube-2k150-cloud-config to kolla-ansible root CA certificate then it works fine.

Temporary Fix is editing the cacert value in cloud-config secret everytime when I create k8s in magnum.

kubectl edit secret -n magnum-system kube-b4rv4-cloud-config

I followed below guides for magnum-cluster-api deployment with kolla-Ansible:

https://satishdotpatel.github.io/openstack-magnum-capi/
https://www.roksblog.de/openstack-magnum-cluster-api-driver/

It could be related to certifi certificates in magnum/kolla but I am not sure how to fix it.

Can someone help here ?

[yaguangtang]

you can add an option in magnum.conf

[drivers]
verify_ca = false

Which will make capi driver to bypass ca verification for your self signed cert.

[rammeena]

I think it is a documentation issue in Kolla-Ansible deployment. This error can be fixed by adding below options in magnum conf:

[capi_client]
region_name = RegionOne
endpoint_type = publicURL
ca_file = /usr/local/share/ca-certificates/kolla-customca-root.crt

[drivers]
openstack_ca_file = /usr/local/share/ca-certificates/kolla-customca-root.crt

[okozachenko1203]

@rammeena Feel free to close the issue if you fixed.

[MaximMonin]

If cluster cannot connect openstack api due to option: ca-file=/etc/config/ca.crt in cloud.conf - it is possible to comment out
[drivers]
#openstack_ca_file = /usr/local/share/ca-certificates/magnum.crt