Objective
Understanding the pod lifecycle is fundamental to effective debugging. When a pod fails to start or terminates unexpectedly, knowing exactly which phase failed — and what component is responsible — cuts triage time dramatically. This exercise creates a realistic pod with all lifecycle hooks configured and traces each phase with kubectl.
Pod Lifecycle Phases
| Phase | Component Responsible | kubectl Signal |
|---|---|---|
| Pending | API Server → etcd | status.phase = Pending |
| Scheduling | kube-scheduler | Event: Scheduled |
| Image Pull | kubelet + container runtime | Event: Pulling/Pulled |
| Container Create | container runtime (containerd) | Event: Created |
| Container Start | kubelet | Event: Started |
| PostStart hook | kubelet | condition: Initialized |
| Readiness probe | kubelet | condition: Ready = True |
| Running | kubelet monitors | status.phase = Running |
| PreStop hook | kubelet | Event: Killing |
| Terminated | kubelet | status.phase = Succeeded/Failed |
Steps
Create a comprehensive pod with all lifecycle hooks
# lifecycle-demo.yaml
cat << 'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: lifecycle-demo
namespace: default
labels:
demo: lifecycle
spec:
initContainers:
- name: init-check
image: busybox:latest
command: ['sh', '-c', 'echo "Init: checking dependencies"; sleep 2; echo "Init: done"']
containers:
- name: app
image: nginx:1.25-alpine
ports:
- containerPort: 80
resources:
requests: {cpu: 50m, memory: 64Mi}
limits: {cpu: 200m, memory: 128Mi}
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo 'PostStart hook executed' >> /tmp/lifecycle.log"]
preStop:
exec:
command: ["/bin/sh", "-c", "echo 'PreStop hook: draining connections'; sleep 5"]
readinessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 3
livenessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 15
periodSeconds: 10
failureThreshold: 3
startupProbe:
httpGet:
path: /
port: 80
failureThreshold: 10
periodSeconds: 3
terminationGracePeriodSeconds: 30
EOFWatch pod status transitions in real time
# Watch phase and conditions kubectl get pod lifecycle-demo -w # Expected progression: # lifecycle-demo 0/1 Pending 0 0s # lifecycle-demo 0/1 Init:0/1 0 1s ← init container # lifecycle-demo 0/1 PodInitializing 0 5s ← init done # lifecycle-demo 0/1 Running 0 7s ← container started # lifecycle-demo 1/1 Running 0 12s ← readiness passed
Examine scheduling decisions
# Check which node the scheduler selected and why kubectl describe pod lifecycle-demo | grep -A5 "Node:" # View scheduler event kubectl get events --field-selector \ involvedObject.name=lifecycle-demo,reason=Scheduled # See the node affinity, tolerations, and resource fit kubectl describe pod lifecycle-demo | \ grep -A20 "Conditions:" | head -20 # Example output: # Conditions: # Type Status # Initialized True ← init containers done # Ready True ← readiness probe passed # ContainersReady True # PodScheduled True ← scheduler assigned node
Verify postStart hook executed
# Check postStart hook left its log entry kubectl exec lifecycle-demo -- cat /tmp/lifecycle.log # Expected: PostStart hook executed # View all events in order kubectl get events \ --field-selector involvedObject.name=lifecycle-demo \ --sort-by='.lastTimestamp' | head -20 # Events show full sequence: # Scheduled, Pulling, Pulled, Created, Started
Examine readiness probe evaluation
# Check probe configuration and current status kubectl describe pod lifecycle-demo | \ grep -A15 "Readiness:" # Temporarily break readiness by changing the endpoint kubectl exec lifecycle-demo -- \ nginx -s stop 2>&1 || true # nginx stops → readiness probe fails → pod marked not-ready sleep 20 kubectl get pod lifecycle-demo # READY column should show 0/1 after nginx stops # Check the probe failure events kubectl get events \ --field-selector involvedObject.name=lifecycle-demo,reason=Unhealthy \ --sort-by='.lastTimestamp'
Observe graceful termination with preStop hook
When a pod is deleted, Kubernetes sends SIGTERM after the preStop hook completes. The container has terminationGracePeriodSeconds to finish in-flight requests.
# Start watching events before deleting kubectl get events \ --field-selector involvedObject.name=lifecycle-demo -w & DELETE_START=$(date -u +%s) # Delete the pod and observe the sequence kubectl delete pod lifecycle-demo # Expected event sequence: # Killing → preStop hook invoked # (5 second sleep in preStop hook) # Container terminated after SIGTERM DELETE_END=$(date -u +%s) echo "Deletion took: $((DELETE_END - DELETE_START))s" # Should be ~5s (preStop) + SIGTERM handling time
Re-create and view full describe output
# Re-create and wait for ready kubectl apply -f lifecycle-demo.yaml kubectl wait pod/lifecycle-demo --for=condition=Ready --timeout=60s # View complete kubectl describe output with annotations kubectl describe pod lifecycle-demo # Key sections to understand: # Node: ← where it's scheduled # Start Time: ← when kubelet started working on it # Labels/Annotations # Status: Running # IP: ← pod IP address # Init Containers: ← init container status # Containers: ← main container status + probe config # Conditions: ← all True when healthy # Volumes: ← projected service account token, etc. # QoS Class: ← Burstable (requests != limits) # Events: ← the lifecycle event log
Success Criteria
Further Reading
- Pod lifecycle — kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle
- Container hooks — kubernetes.io/docs/concepts/containers/container-lifecycle-hooks
- Probes — kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes
- Graceful shutdown — learnk8s.io/graceful-shutdown