Simulate kubernetes pod resisting shutdown
This is a short post about kubernetes pod lifecycle, namely terminating a pod.
When kubernetes shuts down a pod it will:
- Send SIGTERM signal to the container process
- wait for the grace period (default is 30s)
- Send SIGKILL signal to the process
When a Pod is being deleted, it is shown as
Terminatingby some kubectl commands. This
Terminatingstatus is not one of the Pod phases. A Pod is granted a term to terminate gracefully, which defaults to 30 seconds. You can use the flag
--forceto terminate a Pod by force.
- Copied from Pod phase - Pod Lifecycle | Kubernetes
If needed to simulate simulate a container process that ignores the “softer” SIGTERM and has to be shut down using SIGKILL. This is how I did it:
Create pod that ignores SIGTERM
bash.yml file with the following content
apiVersion: apps/v1 kind: Deployment metadata: name: bash labels: app: bash spec: selector: matchLabels: app: bash template: metadata: labels: app: bash spec: containers: - image: ubuntu name: ubuntu command: ["bash", "-c", "trap echo sigterm SIGTERM;while :;do echo hello; sleep 10;done"]
- This is the key part of the template:
trap echo sigterm SIGTERM;while :;do echo hello; sleep 10;done.
- The bash script that “traps” the SIGTERM signal and only prints
- And to keep the container alive, we print
helloevery 10 seconds.
I created deployment with
kubectl apply -f bash.yml.
I deleted that deployment with
kubectl delete deployment bash.
I observed the pod hanging in terminating state.
kubectl get pod NAME READY STATUS RESTARTS AGE bash-7cf45b6b47-8km59 1/1 Terminating 0 22s