Kubernetes CrashLoopBackOff: 7 Causes — 5-Minute Exit Code Diagnosis
K8s_Troubleshooting_Guide, Part 13
You ran kubectl get pods and STATUS shows CrashLoopBackOff, with the RESTARTS count climbing every second? If you landed here mid-incident, remember just one thing: CrashLoopBackOff is not an error message by itself.
Don't panic when you see "back-off restarting failed container"
CrashLoopBackOff is, precisely, this state:
The container starts → exits → starts → exits in a loop, so kubelet stretches the restart interval 10s → 20s → 40s → ... up to a 5-minute cap (that's the back-off) and waits.
In other words, it's only the symptom that "the container keeps dying." Why it dies is something you still have to dig out. That's why the Back-off restarting failed container line in kubectl describe will never tell you the cause. The real clues are the exit code and the logs from just before it died. Follow the steps below in order and you'll have the cause narrowed down in five minutes.
Step 1: Read the exit code first (describe + logs --previous)
The first thing to do is just two commands.
kubectl describe pod <pod-name>
kubectl logs <pod-name> --previous # 죽은 직전 컨테이너의 로그--previous (or -p) is the key. Because the container has already restarted, a plain logs call either shows the newly spawned (not-yet-dead) container or comes back empty. To catch the previous container's dying gasp, you must pass --previous.
In the describe output, look at the Last State block.
Last State: Terminated
Reason: Error
Exit Code: 1 # ← here!
Started: Tue, 17 Jun 2026 09:12:01 +0900
Finished: Tue, 17 Jun 2026 09:12:01 +0900 # Started = Finished → died immediately
Restart Count: 6Look at Reason, Exit Code, and the gap between Started and Finished. If start and finish times are nearly identical, that's the "died immediately, never even booted" pattern.
Exit code mapping table
| Exit Code | Reason | Likely cause | Branch |
|---|---|---|---|
0 | Completed | Exited cleanly but got restarted (main process isn't running, or it backgrounded) | Cause ① |
1 (or 2) | Error | Died from an in-app exception or config error | Causes ②③④⑦ |
137 | OOMKilled / Error | SIGKILL — memory limit exceeded or forced kill | Causes ⑤⑥ |
139 | Error | Segfault (SIGSEGV) — native crash or architecture mismatch | Cause ⑦ |
The exit code is your traffic light. Split it in your head: 0 means "it didn't crash, it finished", 1 means "the app or config is wrong", 137 means "K8s killed it".
Step 2: Diagnose and fix each of the 7 causes (copy-paste ready)
① Main process exits immediately (Exit 0, but it restarts)
- Looks like: Exit Code
0, ReasonCompleted, logs are clean. - Run this:
kubectl logs <pod> -p— confirm it printed normal logs then exited. - Fix it: The container's main process (PID 1) must stay in the foreground. Change CMD so it runs in the foreground, e.g.
nginx -g 'daemon off;'ornode server.js. If you daemonize into the background, PID 1 exits immediately and the container terminates.
② Bad command/args (entrypoint error)
- Looks like:
exec: "xxx": executable file not foundorno such file or directory. - Run this:
Bash
kubectl get pod <pod> -o jsonpath='{.spec.containers[0].command}' kubectl logs <pod> -p - Fix it: Check the manifest for typos in
command/argsand missing absolute paths. If the image is distroless (no shell),["/bin/sh","-c", ...]will not work.
③ Missing env vars, ConfigMap, or Secret
- Looks like: logs show
KeyError,nil pointer,panic: required env DATABASE_URL not set. - Run this:
Bash
kubectl get configmap,secret -n <ns> kubectl describe pod <pod> | grep -A20 Environment - Fix it: Confirm the ConfigMap/Secret name and key you reference actually exist. This is the most common trap when deploying with GitOps (ArgoCD) — the Deployment points at a new ConfigMap that hasn't synced yet, or you changed values without rolling the Pod so the old values stick. Force it with
kubectl rollout restart deploy/<name>.
④ Boot fails because a dependency (DB, Redis) isn't ready
- Looks like:
connection refused,could not connect to db:5432. - Run this:
kubectl get svc,endpoints -n <ns>— check whether the target Service has empty ENDPOINTS. - Fix it: If the app is written to die when the DB isn't there at boot, the 2026 recommended pattern is to wait on the dependency with an initContainer.
Fundamentally, you can also add retry logic in the app and let K8s restart it withYAML
initContainers: - name: wait-for-db image: busybox:1.36 command: ['sh','-c','until nc -z db 5432; do echo waiting; sleep 2; done']restartPolicy: Always.
⑤ Memory limit too low → OOMKilled (Exit 137)
- Looks like: Reason
OOMKilled, Exit Code137. - This isn't the star of this post. If it dies with 137 after running for a while rather than right after boot, it's a memory issue — see 👉 OOMKilled diagnosis (Part 9). Here, just remember the branch: "137 = K8s killed it."
⑥ Forced restart from a failed liveness probe
- Looks like: events show
Liveness probe failed,Killing container. - The app is fine; K8s keeps killing it because of the health-check config. See 👉 Probe failure diagnosis. We'll just take the branch and move on.
⑦ Image/entrypoint error or architecture mismatch
- Looks like:
exec format error, Exit Code139(segfault). - Run this:
kubectl describe pod <pod> | grep Imageto check the tag. - Fix it:
exec format erroris almost always an architecture mismatch. Classic case: you built on Apple Silicon (arm64) and scheduled onto an amd64 node. Multi-arch build withdocker buildx build --platform linux/amd64, or rebuild for the node's architecture. In GitOps environments, also suspect an image tag mismatch where alatesttag cache is still running an old image.
Step 3: 5-minute diagnostic flowchart + debugging tips
flowchart TD
A[CrashLoopBackOff] --> B[kubectl describe + logs -p]
B --> C{Exit Code?}
C -->|0| D[Foreground the main process ①]
C -->|1/2| E{Log message?}
E -->|missing env/config key| F[Check ConfigMap/Secret ③]
E -->|connection refused| G[Dependency / initContainer ④]
E -->|not found/exec error| H[Inspect command/args ②⑦]
C -->|137| I[OOMKilled post / Probe post ⑤⑥]
C -->|139| J[Rebuild for arch / image ⑦]When there are no logs: override with sleep and exec in
The most frustrating case is instant death with no logs at all. Keep the container alive and run commands from inside.
# Temporarily override the entrypoint so the container stays up
spec:
containers:
- name: app
image: myapp:1.2.3
command: ["sleep", "3600"]kubectl exec -it <pod> -- sh
# Run the original command yourself inside the container
/app/start.sh # see the real error message
env | grep DATABASE # confirm env vars were injected
nc -z db 5432 # test connectivity to the dependencyRun the original dying command yourself in a shell and the real error log that K8s swallowed prints right on the screen. This one trick solves a surprising number of cases.
See the full picture with the events timeline
kubectl get events --sort-by=.lastTimestamp -n <ns>LAST TYPE REASON OBJECT MESSAGE
30s Warning BackOff pod/app-xxx Back-off restarting failed container
45s Normal Pulled pod/app-xxx Successfully pulled image "myapp:1.2.3"
50s Warning Failed pod/app-xxx Error: secret "db-cred" not foundSort events by time and the cause-and-effect jumps out — e.g. "image pulled fine, then it blew up because the secret was missing."
Field tip — The two places 1–3 year engineers get stuck on CrashLoopBackOff most often: omitting
--previousand stalling on "logs are empty," and changing a ConfigMap in ArgoCD without rolling the Pod so it keeps running old config. Just watching for those two cuts debug time in half.
Wrap-up: remember three buckets
It looks complicated, but CrashLoopBackOff causes fall into three branches.
- Did the app die? → Exit 0/1, exception in the logs (causes ①②⑦)
- Is the config wrong? → env, ConfigMap, Secret, dependency (causes ③④)
- Did K8s kill it? → 137 OOM, probe failure (causes ⑤⑥)
Read the exit code with describe → pick a branch from the table → copy-paste the matching cause block. That order narrows the scope in five minutes.
Next, Part 14 covers ImagePullBackOff and private-registry auth errors, which often show up alongside CrashLoopBackOff.
References: official docs
The primary source for the behavior, settings, and errors in this post is the official documentation below. Check there for version-specific options and exact behavior.
FAQ
Q. I ran kubectl logs and got nothing.
A. The container has likely already restarted, so you're looking at the new container's logs. Check the previous (dead) container with kubectl logs <pod> --previous. If that's still empty, it died before boot — override the entrypoint with sleep 3600 and kubectl exec in to run it yourself.
Q. Exit Code is 0 — why CrashLoopBackOff?
A. Even a clean exit (0) gets restarted if restartPolicy: Always. The main process needs to stay in the foreground; if it daemonizes or finishes a job and exits, you get this. Keep the process in the foreground.
Q. Are CrashLoopBackOff and ImagePullBackOff different? A. Yes. ImagePullBackOff means the container never even started because the image couldn't be pulled. CrashLoopBackOff means it started but keeps dying. This post covers the latter.
Nodelog는 모든 콘텐츠의 내용과 출처를 공개 전에 검토합니다. 환경(OS·버전)에 따라 결과가 달라질 수 있는 기술 정보는 공식 문서와 함께 확인하며, 검토 기준과 정정 원칙은 편집 정책에서 안내합니다. 오류를 발견하시면 이메일로 제보해 주세요 — 확인 후 신속히 정정합니다.
Comments
Be the first to comment.