How to Fix Kubernetes Back-Off Restarting Failed Container

10 seconds, then 20 seconds, then 40 seconds, eventually capped at five minutes, is Kubernetes’ usual restart back-off when a container repeatedly exits; the message means you need to find why the process stopped, not simply wait for another restart. You will commonly see this as CrashLoopBackOff in kubectl get pods. Start by preserving the failed container’s logs and termination details, then repair the startup command, configuration, dependency, probe, or resource issue that caused the exit.

Use this table to choose the first evidence to collect.

Symptom Likely Cause First Fix
CrashLoopBackOff or Back-off restarting failed container Application process exits, command is invalid, or required configuration is missing Read logs from the previous container instance and inspect the termination reason
OOMKilled The process exceeded its memory limit Investigate memory use, then set realistic resource requests and limits
Liveness probe failed The probe checks the wrong endpoint or starts before the application is usable Correct the probe handler and timing
ContainerCreating with FailedMount A volume path, permission, server export, or credential is wrong Repair the volume configuration before debugging application logs
CreateContainerConfigError A referenced Secret, ConfigMap, environment variable, or volume definition is missing Correct the referenced object or manifest field

What Back-Off Restarting Failed Container Means

A Back-off restarting failed container event means Kubernetes started a container, the container stopped, and Kubernetes is delaying its next restart because repeated immediate restarts would create noise and consume resources.

CrashLoopBackOff is the status typically shown by kubectl get pods. It is not the root error. The useful evidence is usually in one of these places:

  • The container’s standard output and error logs
  • The previous container instance’s logs
  • The Pod Events list
  • The container termination reason and exit code
  • The Pod manifest’s command, args, volumes, environment variables, probes, and resource settings

A restart loop is evidence from the runtime, not the root cause.

Whether Kubernetes restarts a stopped container depends on the workload’s restart policy. Deployments normally run Pods with an Always restart policy, so even a process that exits successfully can be restarted when it was expected to stay running. Jobs use OnFailure or Never: OnFailure restarts a failed container in the Pod, while Never leaves the failed Pod for the Job controller to replace or count as a failed attempt.

How to Fix Back-Off Restarting Failed Container

Identify the failing Pod, inspect its status and previous logs, then correct the command, inputs, or configuration causing restarts.

Step 1: Identify the Failing Pod and Preserve Its Status

Do not delete the Pod as your first action. Deleting it can remove the most useful evidence, especially for a short-lived crash.

  1. List Pods in the workload namespace with the following command, then identify the one with restarts or a CrashLoopBackOff status:
kubectl get pods -n <namespace>
  1. Describe the Pod. This shows container states, restart counts, probe failures, scheduling information, and recent Events.
kubectl describe pod <pod-name> -n <namespace>
  1. Run the following command to retrieve the rendered Pod definition and inspect the actual command, image, environment references, probes, volumes, and limits that Kubernetes received:
kubectl get pod <pod-name> -n <namespace> -o yaml

In the Containers section of kubectl describe, look for:

  • State
  • Last State
  • Reason
  • Exit Code
  • Started
  • Finished

An Exit Code by itself is not enough to diagnose every crash, but it tells you whether Kubernetes successfully launched the process. Events often reveal failures before the process can write any application logs.

Step 2: Read the Previous Container Logs

When a container crashes and restarts quickly, ordinary logs may show output from the current, newly started instance. Use --previous to retrieve output from the instance that actually failed.

kubectl logs <pod-name> -n <namespace> -c <container-name> --previous

Then check the current instance if it stays alive long enough to emit new output.

kubectl logs <pod-name> -n <namespace> -c <container-name>

Use the exact container name shown by kubectl describe pod. This matters for Pods with sidecars: the application container may be healthy while a proxy, log agent, init-like helper, or another sidecar is the one restarting.

Common useful log messages include:

  • command not found
  • Missing module, package, binary, or shared library errors
  • Invalid configuration syntax
  • Authentication or connection failures for a required database or API
  • Permission denied on a mounted file or directory
  • A stack trace from the application runtime
  • A clean application shutdown message that reveals the process was not meant to be a long-running server

If --previous returns no logs, inspect Events and the termination reason. The failure may occur before the process gets far enough to write to standard output or error.

Step 3: Inspect the Startup Command and Its Inputs

A bad container command is one of the fastest ways to create a restart loop. Check the image’s expected entrypoint and the Pod’s command and args fields together; Kubernetes command overrides the image entrypoint, and args overrides its default command arguments.

Pay special attention to shell wrappers. For example, /bin/sh -ec "sleep 300" runs a command string through a shell, and the -e flag tells that shell to exit when a command fails. That is useful when you want a startup script to fail fast, but it also means an otherwise overlooked failing command can terminate the container.

Check for these mistakes:

  1. A binary name that exists on your development machine but not in the image
  2. A script copied to the image without execute permission
  3. An incorrect WORKDIR assumption or a missing relative file path
  4. Incorrect quoting when commands are passed through /bin/sh -c
  5. Environment variables that are present locally but absent in the Pod
  6. A ConfigMap, Secret, certificate, or mounted file that has a different key or path than the application expects
  7. A server command that forks into the background, causing PID 1 to exit and Kubernetes to restart the container

Fix the manifest or image, then apply the corrected workload definition.

kubectl apply -f <manifest-file>.yaml

For an application with a complex startup script, test the same image, command, and required configuration in an isolated environment before rolling it out. I reproduce one failure at a time in a clean container because it removes machine-specific noise.

Step 4: Repair the Specific Failure, Not the Restart Loop

Use the failure evidence to make a narrow correction.

For an application exception, fix the configuration, code, dependency version, credentials, or startup data the logs identify. For a missing executable, install it in the image or correct the command path. For a permission problem, correct ownership, file mode, security context, or the mounted path rather than adding broad permissions.

For an OOMKilled container, review the workload’s memory limit and the application’s actual memory behavior. Raising a limit without understanding the workload can conceal a leak or move pressure to other workloads. Set resource requests and limits that match the application’s expected usage, then verify that the Pod schedules and stays running.

For a failed dependency connection, distinguish between a temporary remote service problem and an application configuration error. Confirm the hostname, port, credentials, TLS settings, network policy, and service readiness. Do not add an infinite retry loop that hides a permanently invalid configuration; log the failing dependency clearly and use deliberate retries where temporary unavailability is expected.

Step 5: Confirm the Corrected Workload Stays Healthy

After applying the fix, wait for the controller to create updated Pods and verify the rollout.

kubectl rollout status deployment/<deployment-name> -n <namespace>

List the Pods in the workload namespace to confirm that replacement Pods were created and to check their restart counts:

kubectl get pods -n <namespace>

A successful repair means the expected containers remain ready, restart counts stop increasing, and logs no longer show the original startup failure. Check the application through its normal health endpoint, service, or test request as well; a Pod that merely stays running is not necessarily serving useful traffic.

For a standalone Pod, Kubernetes will not necessarily recreate it after deletion. For a Deployment, StatefulSet, DaemonSet, or Job, make changes to the controller manifest rather than treating an individual generated Pod as the durable configuration.

Fix Mount Failures and Job-Specific Failures

Investigate FailedMount volume errors and Job deadline or cleanup settings to distinguish startup failures from expected Job termination.

Pod Stuck in ContainerCreating With FailedMount

A Pod in ContainerCreating is not necessarily crash-looping. Kubernetes may be unable to start the container at all because it cannot attach or mount storage.

When Events show FailedMount and mount failed: exit status 32, investigate the volume before inspecting application logs. A mount failure can result from a nonexistent path, an NFS export restriction, invalid credentials, inaccessible storage server, or a permission mismatch.

The William & Mary Kubernetes guide shows an NFS mount failing when the configured server path does not exist, producing a permission-denied mount error. Verify the server name, exported path, PersistentVolume or volume definition, namespace-scoped claim, and access permissions. Apply the corrected manifest and confirm that the Pod progresses beyond ContainerCreating.

Jobs That End Because of a Deadline

A Job is expected to finish, fail, or retry according to its configuration, so do not diagnose every terminated Job as a service-style crash loop.

Check the Job manifest for activeDeadlineSeconds. If it is set to 60, Kubernetes stops that Job when it has not finished within 60 seconds. The correct value is the one in your Job’s activeDeadlineSeconds field and should cover the expected execution time without allowing an unwanted job to run indefinitely.

Also check ttlSecondsAfterFinished before troubleshooting a completed or failed Job. In the William & Mary example, ttlSecondsAfterFinished: 30 removes finished Job resources after about 30 seconds. Increase or remove that cleanup setting temporarily when you need failed Pods and logs to remain available for inspection, then restore an appropriate cleanup policy after debugging.

For Job failures, inspect the Job and its Pods separately.

kubectl describe job <job-name> -n <namespace>
kubectl get pods -n <namespace>

A Job that intentionally uses restartPolicy: Never may produce new failed Pods rather than repeatedly restarting a container in one Pod. Read logs from the specific failed Pod before it is cleaned up.

Fix Probe and Resource Restart Problems

A liveness probe can restart a healthy-but-slow application if it checks too soon, targets the wrong path or port, requires authentication unexpectedly, or has an unrealistic timeout. A readiness probe does not restart containers; it only controls whether a ready endpoint receives traffic.

Start with probe behavior that matches the application lifecycle. KodeKloud’s example uses initialDelaySeconds: 15 and periodSeconds: 10, but these settings must be confirmed in the container’s livenessProbe or readinessProbe YAML against the application’s real startup and response behavior. If startup time varies substantially, use a startup probe so liveness checks do not begin while initialization is still underway.

Do not solve a probe failure by removing all health checks permanently. Instead:

  1. Confirm the application’s actual health endpoint or command.
  2. Ensure the probe uses the correct port, path, protocol, and required headers.
  3. Give startup-dependent applications enough initialization time.
  4. Keep readiness separate from liveness when an application can be alive but temporarily unable to serve requests.
  5. Roll out the change and confirm the probe Events disappear.

If Apache or another web server runs for a long time and then fails, treat it as a runtime failure rather than an initial startup problem. Check the previous container logs, termination reason, active connections or downstream dependency errors, disk and file-permission messages, and whether a liveness probe began failing after the service was already running.

Prevent Future Container Restart Loops

Build prevention into the image, manifest, and deployment process:

  • Test the exact image command that production will run.
  • Log startup configuration safely without printing secrets.
  • Fail clearly when required configuration is missing.
  • Use readiness probes to protect traffic during startup.
  • Use liveness probes only for conditions where restarting is genuinely useful.
  • Set memory requests and limits based on observed workload behavior.
  • Monitor restart counts, failed Job attempts, and container termination reasons.
  • Keep failed Job resources long enough to collect logs when investigating failures.
  • Validate ConfigMap, Secret, volume, and service references before deployment.

A final verification checklist:

  • [ ] The updated controller manifest is applied.
  • [ ] Replacement Pods become ready.
  • [ ] Restart counts remain stable during normal operation.
  • [ ] Previous logs no longer contain the original error.
  • [ ] Events no longer show BackOff, probe failures, mount failures, or configuration errors.
  • [ ] The application responds through its intended service path.

Frequently Asked Questions

How Long Will Kubernetes Wait Before Restarting a Failed Container?

Kubernetes usually waits 10 seconds, then 20 seconds, then 40 seconds, increasing the delay until it reaches a five-minute cap. During the restart delay, inspect previous container logs and the Pod description while the failure evidence is still available.

How Do I Manually Restart a Failed Container?

For a Deployment, restart the workload controller rather than attempting to restart one container directly.

kubectl rollout restart deployment/<deployment-name> -n <namespace>

Then wait for the rollout to finish.

kubectl rollout status deployment/<deployment-name> -n <namespace>

This creates replacement Pods, but it will not fix a bad image, command, probe, secret reference, or resource limit. Correct the root cause first or the new Pods will enter the same restart loop.

Is Back-Off Restarting Failed Container a Docker Error?

No. It is a Kubernetes event produced when Kubernetes is managing a container that repeatedly stops. Docker can run the image underneath Kubernetes, but the Pod definition, controller, restart policy, Events, probes, and resource settings are where you diagnose this specific message.

Why Does kubectl logs --previous Show Nothing?

The container may have failed before it wrote output, the selected container name may be wrong, or no prior instance may exist yet. Run kubectl describe pod <pod-name> -n <namespace> and inspect Events plus Last State for image, mount, Secret, command, and runtime errors.

Why Did My Failed Job Disappear Before I Could Inspect It?

A Job can be deleted automatically after it reaches a terminal state when ttlSecondsAfterFinished is configured. A value of 30 removes finished Job resources after about 30 seconds, so increase or remove that field temporarily when you need time to collect failed Pod logs and Events.

What should you check when an Apache container fails after it has been running?

Check the failed instance’s previous logs and describe the Pod to identify why the container stopped. Review liveness-probe behavior, resource limits, deployment configuration, and required dependencies, since each can cause a container to fail after it has started. Fix the specific error found rather than relying on repeated restarts.