deployments-k8s/examples/interdomain/dns at main · Mixaster995/deployments-k8s

Name	Name	Last commit message	Last commit date
parent directory ..
README.md	README.md

Setup DNS for two clusters

This example shows how to simply configure three k8s clusters to know each other. Can be skipped if clusters setupped with external DNS.

Run

Make sure that we have three KUBECONFIG files.

Check KUBECONFIG1 env:

[[ ! -z $KUBECONFIG1 ]]

Check KUBECONFIG2 env:

[[ ! -z $KUBECONFIG2 ]]

Check KUBECONFIG3 env:

[[ ! -z $KUBECONFIG3 ]]

Get clusters IPs

Switch to cluster1:

export KUBECONFIG=$KUBECONFIG1

Expose kube-dns service:

  kubectl expose service kube-dns -n kube-system --port=53 --target-port=53 --protocol=TCP --name=exposed-kube-dns --type=LoadBalancer

Wait for setting externalIP:

kubectl get services exposed-kube-dns -n kube-system -o go-template='{{index (index (index (index .status "loadBalancer") "ingress") 0) "ip"}}'

Get and store externalIP of the coredns

ip1=$(kubectl get services exposed-kube-dns -n kube-system -o go-template='{{index (index (index (index .status "loadBalancer") "ingress") 0) "ip"}}')
if [[ $ip1 == *"no value"* ]]; then 
    ip1=$(kubectl get services exposed-kube-dns -n kube-system -o go-template='{{index (index (index (index .status "loadBalancer") "ingress") 0) "hostname"}}')
    ip1=$(dig +short $ip1 | head -1)
fi
echo Selected externalIP: $ip1 for cluster1
[[ ! -z $ip1 ]]

Switch to cluster2:

export KUBECONFIG=$KUBECONFIG2

Expose kube-dns service:

kubectl expose service kube-dns -n kube-system --port=53 --target-port=53 --protocol=TCP --name=exposed-kube-dns --type=LoadBalancer

Wait for setting externalIP:

kubectl get services exposed-kube-dns -n kube-system -o go-template='{{index (index (index (index .status "loadBalancer") "ingress") 0) "ip"}}'

Get and store externalIP of the coredns

ip2=$(kubectl get services exposed-kube-dns -n kube-system -o go-template='{{index (index (index (index .status "loadBalancer") "ingress") 0) "ip"}}')
if [[ $ip2 == *"no value"* ]]; then 
    ip2=$(kubectl get services exposed-kube-dns -n kube-system -o go-template='{{index (index (index (index .status "loadBalancer") "ingress") 0) "hostname"}}')
    ip2=$(dig +short $ip2 | head -1)
fi
echo Selected externalIP: $ip2 for cluster1
[[ ! -z $ip2 ]]

Switch to cluster3:

export KUBECONFIG=$KUBECONFIG3

Expose kube-dns service:

kubectl expose service kube-dns -n kube-system --port=53 --target-port=53 --protocol=TCP --name=exposed-kube-dns --type=LoadBalancer

Wait for setting externalIP:

kubectl get services exposed-kube-dns -n kube-system -o go-template='{{index (index (index (index .status "loadBalancer") "ingress") 0) "ip"}}'

Get and store externalIP of the coredns

ip3=$(kubectl get services exposed-kube-dns -n kube-system -o go-template='{{index (index (index (index .status "loadBalancer") "ingress") 0) "ip"}}')
if [[ $ip3 == *"no value"* ]]; then 
    ip3=$(kubectl get services exposed-kube-dns -n kube-system -o go-template='{{index (index (index (index .status "loadBalancer") "ingress") 0) "hostname"}}')
    ip3=$(dig +short $ip3 | head -1)
fi
echo Selected externalIP: $ip3 for cluster1
[[ ! -z $ip3 ]]

Update CoreDNS configmaps:

For the first cluster:

export KUBECONFIG=$KUBECONFIG1

---
cat > configmap.yaml <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns
  namespace: kube-system
data:
  Corefile: |
    .:53 {
        errors
        health {
            lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
            pods insecure
            fallthrough in-addr.arpa ip6.arpa
            ttl 30
        }
        k8s_external my.cluster1
        prometheus :9153
        forward . /etc/resolv.conf {
            max_concurrent 1000
        }
        loop
        reload 5s
    }
    my.cluster2:53 {
      forward . ${ip2}:53 {
        force_tcp
      }
    }
    my.cluster3:53 {
      forward . ${ip3}:53 {
        force_tcp
      }
    }
EOF

Apply CoreDNS config map:

kubectl apply -f configmap.yaml

Also if your cluster coredns is using import plugin it makes sense to use a custom-cordns configmap.

cat > custom-configmap.yaml <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-custom
  namespace: kube-system
data:
  server.override: |
    k8s_external my.cluster1
  proxy2.server: |
    my.cluster2:53 {
      forward . ${ip2}:53 {
        force_tcp
      }
    }
  proxy3.server: |
    my.cluster3:53 {
      forward . ${ip3}:53 {
        force_tcp
      }
    }
EOF

Apply custom CoreDNS config map:

kubectl apply -f custom-configmap.yaml

For the second cluster:

export KUBECONFIG=$KUBECONFIG2

cat > configmap.yaml <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns
  namespace: kube-system
data:
  Corefile: |
    .:53 {
        errors
        health {
            lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
            pods insecure
            fallthrough in-addr.arpa ip6.arpa
            ttl 30
        }
        k8s_external my.cluster2
        prometheus :9153
        forward . /etc/resolv.conf {
            max_concurrent 1000
        }
        loop
        reload 5s
    }
    my.cluster1:53 {
      forward . ${ip1}:53 {
        force_tcp
      }
    }
    my.cluster3:53 {
      forward . ${ip3}:53 {
        force_tcp
      }
    }
EOF

Also if your cluster coredns is using import plugin it makes sense to use a custom-cordns configmap.

cat > custom-configmap.yaml <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-custom
  namespace: kube-system
data:
  server.override: |
    k8s_external my.cluster2
  proxy1.server: |
    my.cluster1:53 {
      forward . ${ip1}:53 {
        force_tcp
      }
    }
  proxy3.server: |
    my.cluster3:53 {
      forward . ${ip3}:53 {
        force_tcp
      }
    }
EOF

Apply custom CoreDNS config map:

kubectl apply -f custom-configmap.yaml

Apply CoreDNS config map:

kubectl apply -f configmap.yaml

For the third cluster:

export KUBECONFIG=$KUBECONFIG3

cat > configmap.yaml <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns
  namespace: kube-system
data:
  Corefile: |
    .:53 {
        errors
        health {
            lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
            pods insecure
            fallthrough in-addr.arpa ip6.arpa
            ttl 30
        }
        k8s_external my.cluster3
        prometheus :9153
        forward . /etc/resolv.conf {
            max_concurrent 1000
        }
        loop
        reload 5s
    }
    my.cluster1:53 {
      forward . ${ip1}:53 {
        force_tcp
      }
    }
    my.cluster2:53 {
      forward . ${ip2}:53 {
        force_tcp
      }
    }
EOF

Apply CoreDNS config map:

kubectl apply -f configmap.yaml

Also if your cluster coredns is using import plugin it makes sense to use a custom-coredns configmap.

cat > custom-configmap.yaml <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-custom
  namespace: kube-system
data:
  server.override: |
    k8s_external my.cluster3
  proxy1.server: |
    my.cluster1:53 {
      forward . ${ip1}:53 {
        force_tcp
      }
    }
  proxy2.server: |
    my.cluster2:53 {
      forward . ${ip2}:53 {
        force_tcp
      }
    }
EOF

Apply custom CoreDNS config map:

kubectl apply -f custom-configmap.yaml

Cleanup

export KUBECONFIG=$KUBECONFIG1 && kubectl delete service -n kube-system exposed-kube-dns
export KUBECONFIG=$KUBECONFIG2 && kubectl delete service -n kube-system exposed-kube-dns
export KUBECONFIG=$KUBECONFIG3 && kubectl delete service -n kube-system exposed-kube-dns

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dns

dns

README.md

Setup DNS for two clusters

Run

Cleanup

Files

dns

Directory actions

More options

Directory actions

More options

Latest commit

History

dns

Folders and files

parent directory

README.md

Setup DNS for two clusters

Run

Cleanup