kubectl get endpoints <none>: Diagnose Service connection refused in 5 minutes

kubectl get endpoints is empty (<none>)? A 5-minute diagnostic guide for Service connection refused / no endpoints

K8s_Troubleshooting_Guide Part 14

You created a Service, ran curl service-name:80, and got connection refused or an endless timeout. The Pod is clearly Running, but traffic never arrives. This post is for that stuck moment.

To cut to the chase: 90% of these cases happen because the list of Pods the Service should send traffic to (Endpoints) is empty. A Service is just a virtual IP; the actual list of Pod addresses that receive packets lives in Endpoints (or EndpointSlice). If that list is empty, traffic has nowhere to go and is refused.

This post focuses on cases where the name resolves (DNS is fine) but endpoints are empty. CoreDNS name-resolution failures are covered in a separate installment.

Run these three lines first: the primary diagnostic trio

Don't vaguely guess at the symptom. Narrow the scope in five minutes with the three commands below.

# ① 가장 먼저 — Endpoints가 비어있는가?
kubectl get endpoints my-svc

# ② Service의 selector와 포트 설정 확인
kubectl describe svc my-svc

# ③ Pod의 실제 label과 READY 상태 확인
kubectl get pods --show-labels

① The get endpoints output is the fork in the road.

# 비정상 — 여기가 문제의 시작
NAME     ENDPOINTS   AGE
my-svc   <none>      3m       ← ★ Endpoints가 비어있음 → 3장으로

# 정상 — IP가 채워져 있음
NAME     ENDPOINTS                         AGE
my-svc   10.244.1.7:80,10.244.2.4:80       3m   ← endpoint는 있는데 안 됨 → 4장으로

• If you see <none> → selector ↔ label mismatch or Readiness failure (section 3 below)
• If IPs are populated but there's no response → targetPort typo / Service type mix-up / kube-proxy (section 4)

② What to read in describe svc:

Selector:          app=backend,tier=api    ← 이 라벨을 가진 Pod만 endpoint에 등록됨
Port:              80/TCP                   ← 클라이언트 접속 포트
TargetPort:        8080/TCP                 ← Pod로 전달할 포트 (★ 오타 단골)
Endpoints:         <none>                   ← 다시 한번 확인 사살

③ What to read in get pods --show-labels: Eyeball the READY column and LABELS against the Selector.

Endpoints are empty (<none>): selector, labels, and Readiness

Cause 1 — selector ↔ Pod label mismatch

This is the most common culprit. If zero Pods have labels that exactly match the Service selector, endpoints stay empty forever. Don't guess—confirm it with commands.

# Service가 찾고 있는 selector를 그대로 추출
kubectl get svc my-svc -o jsonpath='{.spec.selector}'
# → {"app":"backend","tier":"api"}

# 그 selector로 실제 매칭되는 Pod 수를 직접 세어본다
kubectl get pods -l app=backend,tier=api
# → No resources found.   ★ 0개면 selector 불일치 확정!

If you get No resources found, the culprit is confirmed. Either the Pod is labeled tier=backend while the Service is looking for tier=api, or there's a typo.

before / after YAML

# ❌ before — selector 오타 (tier 값이 Pod와 다름)
apiVersion: v1
kind: Service
metadata:
  name: my-svc
spec:
  selector:
    app: backend
    tier: api        # ← Pod의 실제 라벨은 tier: backend
  ports:
    - port: 80
      targetPort: 80

# ✅ after — Pod 라벨과 정확히 일치
spec:
  selector:
    app: backend
    tier: backend    # ← 수정

kubectl apply -f my-svc.yaml
kubectl get endpoints my-svc
# NAME     ENDPOINTS                     AGE
# my-svc   10.244.1.7:80,10.244.2.4:80   ← 채워졌다!

Cause 2 — Dropped from endpoints due to Readiness Probe failure

If the selector is correct but you still see <none>, suspect a Pod that is Running but READY 0/1. Kubernetes automatically excludes Pods that fail the Readiness Probe from the endpoint list. That's correct behavior for safely blocking traffic, but a misconfigured probe produces the "Pod is up, Service comes up empty" situation.

kubectl get pods
# NAME            READY   STATUS    RESTARTS   AGE
# backend-xxxxx   0/1     Running   0          2m   ★ Running인데 READY 0/1

kubectl describe pod backend-xxxxx
# Events:
#   Warning  Unhealthy  ...  Readiness probe failed: HTTP probe failed
#                            with statuscode: 404   ← probe path/port 오타

A common mistake is a probe path or port that doesn't match the actual app. For example, if the app serves /healthz on 8080 but the probe hits /health on 8000, you'll get perpetual 404/refused and the Pod never registers as an endpoint. Align the probe path and port with the app's real values, and shortly after you'll see READY 1/1 and endpoints fill in.

One line from the field: Seeing empty endpoints for about 5 seconds right after a new deploy is normal (waiting for the first probe to pass). But if it's still <none> after 1–2 minutes, it's almost certainly a config problem. The habit of watching with watch kubectl get endpoints my-svc cuts debugging time in half.

When endpoints are populated but there's no response

If you can see IPs in endpoints but curl still fails, suspect the port or Service type along the traffic path.

Cause 3 — targetPort typo (telling the three ports apart)

This is the most confusing spot. Ports appear in three places, each with a different meaning.

The key rule: targetPort must match containerPort (the port the app actually listens on). If nginx listens on 80 but you set targetPort: 8080, endpoints will point at 10.244.x.x:8080, nobody is listening there, and you get connection refused.

# ❌ before — targetPort가 컨테이너 리슨 포트와 불일치
spec:
  ports:
    - port: 80
      targetPort: 8080   # ← nginx는 80을 리슨하는데 8080을 가리킴

# ✅ after
spec:
  ports:
    - port: 80
      targetPort: 80     # ← containerPort와 일치

kubectl apply -f my-svc.yaml
kubectl get endpoints my-svc
# my-svc   10.244.1.7:80,10.244.2.4:80   ← 포트가 :80으로 바뀜

Cause 4 — Service type mix-up + isolated in-cluster test

ClusterIP (the default) is reachable only from inside the cluster. To connect from outside, you need NodePort (exposed on a fixed port of the node IP) or LoadBalancer (an external LB is assigned). "curl to ClusterIP from my laptop doesn't work" is often expected behavior.

To check whether it's a real problem, test in isolation from inside the cluster.

kubectl run tmp --rm -it --image=busybox -- wget -qO- my-svc:80

If you get a response here, the Service itself is fine—you only need to fix the external access path (type/Ingress).

Cause 5 — kube-proxy / iptables (last check)

If endpoints look healthy, labels match, and ports are correct, look at the node's traffic-handling layer.

kubectl -n kube-system get pods -l k8s-app=kube-proxy

If kube-proxy is in CrashLoopBackOff, or the iptables/IPVS rules that send ClusterIP traffic to actual Pods are missing on the node, packets get lost even with healthy endpoints. (This deeper area is covered in a separate post.)

Bonus for modern clusters: EndpointSlice and service mesh

Recent clusters use EndpointSlice instead of Endpoints by default. Check it as well.

kubectl get endpointslices -l kubernetes.io/service-name=my-svc

Also remember that in service mesh / eBPF environments like Istio / Cilium, an eBPF dataplane handles traffic instead of kube-proxy, so the diagnostic point shifts from iptables checks to the sidecar/CNI layer.

Conclusion: a 5-minute checklist by cause

Eliminate causes in order and you'll always catch the culprit.

1. kubectl get endpoints my-svc → is it <none>, or populated?
2. If <none> → check whether get svc -o jsonpath='{.spec.selector}' vs get pods -l ... matches 0 Pods (selector mismatch)
3. Selector is correct but still <none> → check READY 0/1 in get pods + Readiness probe failed in describe pod
4. Populated but still failing → confirm the three ports port / targetPort / containerPort match
5. → Service type (ClusterIP is internal-only) + isolated in-cluster test with kubectl run tmp
6. → If still failing, check kube-proxy status + EndpointSlice

References: official docs

The primary sources for the behavior, settings, and errors covered in this post are the official docs below. Check them for version-specific options and exact behavior.

• Kubernetes official documentation

Frequently asked questions (FAQ)

Q. The Pod is Running but endpoints are empty. Why? A. Look at the READY column. If it's 0/1, the Readiness Probe hasn't passed, and Kubernetes automatically excludes that Pod from endpoints. Align the probe path/port with the app's actual values and they'll fill in.

Q. The selector looks correct but I have 0 endpoints. How do I confirm? A. Count them directly with kubectl get pods -l <selector>. If you get No resources found, a label mismatch is confirmed. Don't compare by eye—paste the selector value into -l and verify with a command.

Q. Endpoints show IPs but curl gets connection refused. A. Check whether targetPort differs from the port the container actually listens on. If the port after the endpoint address (:8080, etc.) doesn't match the app listen port, the connection is refused. Also, ClusterIP only works inside the cluster, so start with an in-cluster test: kubectl run tmp --rm -it --image=busybox -- wget -qO- my-svc:80.

Coming next: Ingress 404/503 — debugging the path from request to backend. We'll trace cases where the Service is fine but Ingress is blocking.