/인프라/Kubernetes CrashLoopBackOff: 7 Causes — Diagnose in 5 Minutes with Exit Codes
InfrastructureCrashLoopBackOffKubernetes 트러블슈팅

Kubernetes CrashLoopBackOff: 7 Causes — Diagnose in 5 Minutes with Exit Codes

When a Pod is stuck in CrashLoopBackOff, read the exit code (0/1/137/139) with describe and logs --previous to narrow the cause in five minutes. Copy-paste commands for each of the seven causes plus a flowchart so you can fix it on the spot

Kubernetes CrashLoopBackOff: 7 Causes — Diagnose in 5 Minutes with Exit Codes

Kubernetes CrashLoopBackOff: 7 Causes — 5-Minute Exit Code Diagnosis

K8s_Troubleshooting_Guide, Part 13

You ran kubectl get pods and STATUS shows CrashLoopBackOff, with the RESTARTS count climbing every second? If you landed here mid-incident, remember just one thing: CrashLoopBackOff is not an error message by itself.

Don't panic when you see "back-off restarting failed container"

CrashLoopBackOff is, precisely, this state:

The container starts → exits → starts → exits in a loop, so kubelet stretches the restart interval 10s → 20s → 40s → ... up to a 5-minute cap (that's the back-off) and waits.

In other words, it's only the symptom that "the container keeps dying." Why it dies is something you still have to dig out. That's why the Back-off restarting failed container line in kubectl describe will never tell you the cause. The real clues are the exit code and the logs from just before it died. Follow the steps below in order and you'll have the cause narrowed down in five minutes.

Step 1: Read the exit code first (describe + logs --previous)

The first thing to do is just two commands.

Bash
kubectl describe pod <pod-name>
kubectl logs <pod-name> --previous   # 죽은 직전 컨테이너의 로그

--previous (or -p) is the key. Because the container has already restarted, a plain logs call either shows the newly spawned (not-yet-dead) container or comes back empty. To catch the previous container's dying gasp, you must pass --previous.

In the describe output, look at the Last State block.

TEXT
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1          # ← here!
      Started:      Tue, 17 Jun 2026 09:12:01 +0900
      Finished:     Tue, 17 Jun 2026 09:12:01 +0900   # Started = Finished → died immediately
    Restart Count:  6

Look at Reason, Exit Code, and the gap between Started and Finished. If start and finish times are nearly identical, that's the "died immediately, never even booted" pattern.

Exit code mapping table

Exit CodeReasonLikely causeBranch
0CompletedExited cleanly but got restarted (main process isn't running, or it backgrounded)Cause ①
1 (or 2)ErrorDied from an in-app exception or config errorCauses ②③④⑦
137OOMKilled / ErrorSIGKILL — memory limit exceeded or forced killCauses ⑤⑥
139ErrorSegfault (SIGSEGV) — native crash or architecture mismatchCause ⑦

The exit code is your traffic light. Split it in your head: 0 means "it didn't crash, it finished", 1 means "the app or config is wrong", 137 means "K8s killed it".

Step 2: Diagnose and fix each of the 7 causes (copy-paste ready)

① Main process exits immediately (Exit 0, but it restarts)

  • Looks like: Exit Code 0, Reason Completed, logs are clean.
  • Run this: kubectl logs <pod> -p — confirm it printed normal logs then exited.
  • Fix it: The container's main process (PID 1) must stay in the foreground. Change CMD so it runs in the foreground, e.g. nginx -g 'daemon off;' or node server.js. If you daemonize into the background, PID 1 exits immediately and the container terminates.

② Bad command/args (entrypoint error)

  • Looks like: exec: "xxx": executable file not found or no such file or directory.
  • Run this:
    Bash
    kubectl get pod <pod> -o jsonpath='{.spec.containers[0].command}'
    kubectl logs <pod> -p
  • Fix it: Check the manifest for typos in command/args and missing absolute paths. If the image is distroless (no shell), ["/bin/sh","-c", ...] will not work.

③ Missing env vars, ConfigMap, or Secret

  • Looks like: logs show KeyError, nil pointer, panic: required env DATABASE_URL not set.
  • Run this:
    Bash
    kubectl get configmap,secret -n <ns>
    kubectl describe pod <pod> | grep -A20 Environment
  • Fix it: Confirm the ConfigMap/Secret name and key you reference actually exist. This is the most common trap when deploying with GitOps (ArgoCD) — the Deployment points at a new ConfigMap that hasn't synced yet, or you changed values without rolling the Pod so the old values stick. Force it with kubectl rollout restart deploy/<name>.

④ Boot fails because a dependency (DB, Redis) isn't ready

  • Looks like: connection refused, could not connect to db:5432.
  • Run this: kubectl get svc,endpoints -n <ns> — check whether the target Service has empty ENDPOINTS.
  • Fix it: If the app is written to die when the DB isn't there at boot, the 2026 recommended pattern is to wait on the dependency with an initContainer.
    YAML
    initContainers:
      - name: wait-for-db
        image: busybox:1.36
        command: ['sh','-c','until nc -z db 5432; do echo waiting; sleep 2; done']
    Fundamentally, you can also add retry logic in the app and let K8s restart it with restartPolicy: Always.

⑤ Memory limit too low → OOMKilled (Exit 137)

  • Looks like: Reason OOMKilled, Exit Code 137.
  • This isn't the star of this post. If it dies with 137 after running for a while rather than right after boot, it's a memory issue — see 👉 OOMKilled diagnosis (Part 9). Here, just remember the branch: "137 = K8s killed it."

⑥ Forced restart from a failed liveness probe

  • Looks like: events show Liveness probe failed, Killing container.
  • The app is fine; K8s keeps killing it because of the health-check config. See 👉 Probe failure diagnosis. We'll just take the branch and move on.

⑦ Image/entrypoint error or architecture mismatch

  • Looks like: exec format error, Exit Code 139 (segfault).
  • Run this: kubectl describe pod <pod> | grep Image to check the tag.
  • Fix it: exec format error is almost always an architecture mismatch. Classic case: you built on Apple Silicon (arm64) and scheduled onto an amd64 node. Multi-arch build with docker buildx build --platform linux/amd64, or rebuild for the node's architecture. In GitOps environments, also suspect an image tag mismatch where a latest tag cache is still running an old image.

Step 3: 5-minute diagnostic flowchart + debugging tips

MERMAID
flowchart TD
  A[CrashLoopBackOff] --> B[kubectl describe + logs -p]
  B --> C{Exit Code?}
  C -->|0| D[Foreground the main process ①]
  C -->|1/2| E{Log message?}
  E -->|missing env/config key| F[Check ConfigMap/Secret ③]
  E -->|connection refused| G[Dependency / initContainer ④]
  E -->|not found/exec error| H[Inspect command/args ②⑦]
  C -->|137| I[OOMKilled post / Probe post ⑤⑥]
  C -->|139| J[Rebuild for arch / image ⑦]

When there are no logs: override with sleep and exec in

The most frustrating case is instant death with no logs at all. Keep the container alive and run commands from inside.

YAML
# Temporarily override the entrypoint so the container stays up
spec:
  containers:
    - name: app
      image: myapp:1.2.3
      command: ["sleep", "3600"]
Bash
kubectl exec -it <pod> -- sh
# Run the original command yourself inside the container
/app/start.sh        # see the real error message
env | grep DATABASE  # confirm env vars were injected
nc -z db 5432        # test connectivity to the dependency

Run the original dying command yourself in a shell and the real error log that K8s swallowed prints right on the screen. This one trick solves a surprising number of cases.

See the full picture with the events timeline

Bash
kubectl get events --sort-by=.lastTimestamp -n <ns>
TEXT
LAST   TYPE      REASON      OBJECT        MESSAGE
30s    Warning   BackOff     pod/app-xxx   Back-off restarting failed container
45s    Normal    Pulled      pod/app-xxx   Successfully pulled image "myapp:1.2.3"
50s    Warning   Failed      pod/app-xxx   Error: secret "db-cred" not found

Sort events by time and the cause-and-effect jumps out — e.g. "image pulled fine, then it blew up because the secret was missing."

Field tip — The two places 1–3 year engineers get stuck on CrashLoopBackOff most often: omitting --previous and stalling on "logs are empty," and changing a ConfigMap in ArgoCD without rolling the Pod so it keeps running old config. Just watching for those two cuts debug time in half.

Wrap-up: remember three buckets

It looks complicated, but CrashLoopBackOff causes fall into three branches.

  • Did the app die? → Exit 0/1, exception in the logs (causes ①②⑦)
  • Is the config wrong? → env, ConfigMap, Secret, dependency (causes ③④)
  • Did K8s kill it? → 137 OOM, probe failure (causes ⑤⑥)

Read the exit code with describe → pick a branch from the table → copy-paste the matching cause block. That order narrows the scope in five minutes.

Next, Part 14 covers ImagePullBackOff and private-registry auth errors, which often show up alongside CrashLoopBackOff.

References: official docs

The primary source for the behavior, settings, and errors in this post is the official documentation below. Check there for version-specific options and exact behavior.

FAQ

Q. I ran kubectl logs and got nothing. A. The container has likely already restarted, so you're looking at the new container's logs. Check the previous (dead) container with kubectl logs <pod> --previous. If that's still empty, it died before boot — override the entrypoint with sleep 3600 and kubectl exec in to run it yourself.

Q. Exit Code is 0 — why CrashLoopBackOff? A. Even a clean exit (0) gets restarted if restartPolicy: Always. The main process needs to stay in the foreground; if it daemonizes or finishes a job and exits, you get this. Keep the process in the foreground.

Q. Are CrashLoopBackOff and ImagePullBackOff different? A. Yes. ImagePullBackOff means the container never even started because the image couldn't be pulled. CrashLoopBackOff means it started but keeps dying. This post covers the latter.

확인 정보
✦ ✦ ✦
편집 검토 · Editorial Review

Nodelog는 모든 콘텐츠의 내용과 출처를 공개 전에 검토합니다. 환경(OS·버전)에 따라 결과가 달라질 수 있는 기술 정보는 공식 문서와 함께 확인하며, 검토 기준과 정정 원칙은 편집 정책에서 안내합니다. 오류를 발견하시면 이메일로 제보해 주세요 — 확인 후 신속히 정정합니다.

편집 책임 · Nodelog 기술 편집팀·발행 · ·업데이트 ·
관련 공식 문서Kubernetes 공식 문서

Comments

Be the first to comment.