This page provides hints on diagnosing DNS problems.
You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube, or you can use one of these Kubernetes playgrounds:
To check the version, enter
Create a file named busybox.yaml with the following contents:
Then create a pod using this file and verify its status:
$ kubectl create -f busybox.yaml pod "busybox" created $ kubectl get pods busybox NAME READY STATUS RESTARTS AGE busybox 1/1 Running 0 <some-time>
Once that pod is running, you can exec
nslookup in that environment.
If you see something like the following, DNS is working correctly.
$ kubectl exec -ti busybox -- nslookup kubernetes.default Server: 10.0.0.10 Address 1: 10.0.0.10 Name: kubernetes.default Address 1: 10.0.0.1
nslookup command fails, check the following:
$ kubectl exec busybox cat /etc/resolv.conf
Verify that the search path and name server are set up like the following (note that search path may vary for different cloud providers):
search default.svc.cluster.local svc.cluster.local cluster.local google.internal c.gce_project_id.internal nameserver 10.0.0.10 options ndots:5
Errors such as the following indicate a problem with the kube-dns add-on or associated Services:
$ kubectl exec -ti busybox -- nslookup kubernetes.default Server: 10.0.0.10 Address 1: 10.0.0.10 nslookup: can't resolve 'kubernetes.default'
$ kubectl exec -ti busybox -- nslookup kubernetes.default Server: 10.0.0.10 Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local nslookup: can't resolve 'kubernetes.default'
kubectl get pods command to verify that the DNS pod is running.
$ kubectl get pods --namespace=kube-system -l k8s-app=kube-dns NAME READY STATUS RESTARTS AGE ... kube-dns-v19-ezo1y 3/3 Running 0 1h ...
If you see that no pod is running or that the pod has failed/completed, the DNS add-on may not be deployed by default in your current environment and you will have to deploy it manually.
kubectl logs command to see logs for the DNS daemons.
$ kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns $ kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq $ kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c sidecar
See if there is any suspicious log. Letter ‘
F’ at the beginning
represent Warning, Error and Failure. Please search for entries that have these
as the logging level and use
to report unexpected errors.
Verify that the DNS service is up by using the
kubectl get service command.
$ kubectl get svc --namespace=kube-system NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE ... kube-dns 10.0.0.10 <none> 53/UDP,53/TCP 1h ...
If you have created the service or in the case it should be created by default but it does not appear, see debugging services for more information.
You can verify that DNS endpoints are exposed by using the
kubectl get endpoints
$ kubectl get ep kube-dns --namespace=kube-system NAME ENDPOINTS AGE kube-dns 10.180.3.17:53,10.180.3.17:53 1h
If you do not see the endpoints, see endpoints section in the debugging services documentation.
For additional Kubernetes DNS examples, see the cluster-dns examples in the Kubernetes GitHub repository.
Kubernetes installs do not configure the nodes’ resolv.conf files to use the cluster DNS by default, because that process is inherently distro-specific. This should probably be implemented eventually.
Linux’s libc is impossibly stuck (see this bug from
2005) with limits of just
nameserver records and 6 DNS
search records. Kubernetes needs to
nameserver record and 3
search records. This means that if a
local installation already uses 3
nameservers or uses more than 3
some of those settings will be lost. As a partial workaround, the node can run
dnsmasq which will provide more
nameserver entries, but not more
entries. You can also use kubelet’s
If you are using Alpine version 3.3 or earlier as your base image, DNS may not work properly owing to a known issue with Alpine. Check here for more information.
Release 1.3 introduced Cluster Federation support for multi-site Kubernetes installations. This required some minor (backward-compatible) changes to the way the Kubernetes cluster DNS server processes DNS queries, to facilitate the lookup of federated services (which span multiple Kubernetes clusters). See the Cluster Federation Administrators’ Guide for more details on Cluster Federation and multi-site support.