kubernetes故障之pod迁移

kubernetes故障之pod迁移

kubernetes故障之pod迁移

kubernetes故障之pod迁移

相关操作命令

# 查看节点
kubectl get nodes

# 设置节点为不可调度
kubectl cordon <NodeName>

# 设置节点为可调度
kubectl uncordon <NodeName>

# pod漂移到可调度的node节点上
kubectl drain <不可调度的node,上面有pod> --force --ignore-daemonsets

实际操作,把”cn-hangzhou.i-bp1bt6np98dbi2xmno0o,cn-hangzhou.i-bp1bt6np98dbi2xmno0p”设置为不可调度

# 查看node节点
kube-shell> kubectl get nodes
NAME                                 STATUS   ROLES    AGE   VERSION
cn-hangzhou.i-bp1bt6np98dbi2xmno0o   Ready    <none>   24d   v1.12.6-aliyun.1
cn-hangzhou.i-bp1bt6np98dbi2xmno0p   Ready    <none>   24d   v1.12.6-aliyun.1
cn-hangzhou.i-bp1cu5nvk55l2usiufx5   Ready    <none>   35m   v1.12.6-aliyun.1
kube-shell> 

# 设置node节点为不可调度
kube-shell> kubectl cordon cn-hangzhou.i-bp1bt6np98dbi2xmno0o
node/cn-hangzhou.i-bp1bt6np98dbi2xmno0o cordoned
kube-shell>  kubectl cordon cn-hangzhou.i-bp1bt6np98dbi2xmno0p
node/cn-hangzhou.i-bp1bt6np98dbi2xmno0p cordoned
kube-shell>

kubectl drain命令漂移不可调度node上的pod节点

# 查看修改调度后的node节点信息,其中2个node不可调度
kube-shell> kubectl get nodes
NAME                                 STATUS                     ROLES    AGE   VERSION
cn-hangzhou.i-bp1bt6np98dbi2xmno0o   Ready,SchedulingDisabled   <none>   24d   v1.12.6-aliyun.1
cn-hangzhou.i-bp1bt6np98dbi2xmno0p   Ready,SchedulingDisabled   <none>   24d   v1.12.6-aliyun.1
cn-hangzhou.i-bp1cu5nvk55l2usiufx5   Ready                      <none>   37m   v1.12.6-aliyun.1
kube-shell>

# 查看pod信息在那个node节点上
kube-shell> kubectl get pods -o wide
NAME                        READY   STATUS    RESTARTS   AGE   IP             NODE                                 NOMINATED NODE
mynginx1-5dd8d4d78b-wgfjw   1/1     Running   0          15d   172.24.0.142   cn-hangzhou.i-bp1bt6np98dbi2xmno0o   <none>
kube-shell>

# 把在node节点"cn-hangzhou.i-bp1bt6np98dbi2xmno0o"不可以node上的pod调度到可用的pod上
kube-shell> kubectl drain cn-hangzhou.i-bp1bt6np98dbi2xmno0o --force --ignore-daemonsets
node/cn-hangzhou.i-bp1bt6np98dbi2xmno0o already cordoned
WARNING: Ignoring DaemonSet-managed pods: node-exporter-v5knf, cattle-node-agent-rlb2d, flexvolume-25spk, kube-flannel-ds-6p557, kube-proxy-worker-fbdj8
pod/mynginx1-5dd8d4d78b-wgfjw evicted
pod/coredns-67fc48b9d7-f88lb evicted
pod/cattle-cluster-agent-57bd559c78-ck2mm evicted
pod/kube-state-metrics-b685fc8c4-tkzxt evicted
kube-shell>

验证

# 查看这个pod是不是已经漂移到可用的node上
kube-shell> kubectl get nodes
NAME                                 STATUS                     ROLES    AGE   VERSION
cn-hangzhou.i-bp1bt6np98dbi2xmno0o   Ready,SchedulingDisabled   <none>   24d   v1.12.6-aliyun.1
cn-hangzhou.i-bp1bt6np98dbi2xmno0p   Ready,SchedulingDisabled   <none>   24d   v1.12.6-aliyun.1
cn-hangzhou.i-bp1cu5nvk55l2usiufx5   Ready                      <none>   55m   v1.12.6-aliyun.1
kube-shell> kubectl get pods -o wide
NAME                             READY   STATUS              RESTARTS   AGE     IP           NODE                                 NOMINATED NODE
mynginx1-5dd8d4d78b-wtdb8        1/1     Running             0          21m     172.24.1.3   cn-hangzhou.i-bp1cu5nvk55l2usiufx5   <none>
kube-shell>

参考文档

Safely Drain a Node while Respecting the PodDisruptionBudget - Kubernetes

https://kubernetes.io/docs/tasks/run-application/configure-pdb/

https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#drain


See also