查看日志
[root@iZbp1c4tqwd9kaykklejmxZ ~]# journalctl -u kubelet -f
-- Logs begin at Fri 2019-10-04 11:50:57 CST. --
Nov 01 13:28:21 iZbp1c4tqwd9kaykklejmxZ kubelet[9731]: E1101 13:28:21.575137 9731 kubelet_volumes.go:154] Orphaned pod "bb5c8fa3-b4b1-11e9-96a8-0a1877e1c33d" found, but volume paths are still present on disk : There were a total of 1 errors similar to this. Turn up verbosity to see them.
Nov 01 13:28:22 iZbp1c4tqwd9kaykklejmxZ kubelet[9731]: W1101 13:28:22.826291 9731 reflector.go:270] object-"monitoring"/"grafana-dashboard-statefulset": watch of *v1.ConfigMap ended with: too old resource version: 2345008003 (2345014010)
Nov 01 13:28:23 iZbp1c4tqwd9kaykklejmxZ kubelet[9731]: E1101 13:28:23.564270 9731 kubelet_volumes.go:154] Orphaned pod "bb5c8fa3-b4b1-11e9-96a8-0a1877e1c33d" found, but volume paths are still present on disk : There were a total of 1 errors similar to this. Turn up verbosity to see them.
Nov 01 13:28:25 iZbp1c4tqwd9kaykklejmxZ kubelet[9731]: E1101 13:28:25.572723 9731 kubelet_volumes.go:154] Orphaned pod "bb5c8fa3-b4b1-11e9-96a8-0a1877e1c33d" found, but volume paths are still present on disk : There were a total of 1 errors similar to this. Turn up verbosity to see them.
Nov 01 13:28:27 iZbp1c4tqwd9kaykklejmxZ kubelet[9731]: E1101 13:28:27.564780 9731 kubelet_volumes.go:154] Orphaned pod "bb5c8fa3-b4b1-11e9-96a8-0a1877e1c33d" found, but volume paths are still present on disk : There were a total of 1 errors similar to this. Turn up verbosity to see them.
Nov 01 13:28:29 iZbp1c4tqwd9kaykklejmxZ kubelet[9731]: E1101 13:28:29.579137 9731 kubelet_volumes.go:154] Orphaned pod "bb5c8fa3-b4b1-11e9-96a8-0a1877e1c33d" found, but volume paths are still present on disk : There were a total of 1 errors similar to this. Turn up verbosity to see them.
Nov 01 13:28:31 iZbp1c4tqwd9kaykklejmxZ kubelet[9731]: E1101 13:28:31.564101 9731 kubelet_volumes.go:154] Orphaned pod "bb5c8fa3-b4b1-11e9-96a8-0a1877e1c33d" found, but volume paths are still present on disk : There were a total of 1 errors similar to this. Turn up verbosity to see them.
查看pods目录
[root@iZbp1c4tqwd9kaykklejmxZ ~]# ll /var/lib/kubelet/pods/bb5c8fa3-b4b1-11e9-96a8-0a1877e1c33d/
total 16
drwxr-x--- 4 root root 4096 Aug 2 07:11 containers
-rw-r--r-- 1 root root 234 Aug 2 07:11 etc-hosts
drwxr-x--- 3 root root 4096 Aug 2 07:11 plugins
drwxr-x--- 5 root root 4096 Aug 2 07:11 volumes
查看etc-hosts文件
[root@iZbp1c4tqwd9kaykklejmxZ ~]# cat /var/lib/kubelet/pods/b5750cd8-b4b1-11e9-96a8-0a1877e1c33d/etc-hosts
# Kubernetes-managed hosts file.
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
fe00::0 ip6-mcastprefix
fe00::1 ip6-allnodes
fe00::2 ip6-allrouters
172.22.3.153 hqhz-newsds-prod-799c4657d9-r5259
[root@iZbp1c4tqwd9kaykklejmxZ ~]#
使用kubectl查看pod是否存在
kube-shell> kubectl get pods -o wide | grep b5750cd8-b4b1-11e9-96a8-0a1877e1c33d
kube-shell>
删除报错的不存在的pod
[root@iZbp1c4tqwd9kaykklejmxZ ~]# rm -rf /var/lib/kubelet/pods/bb5c8fa3-b4b1-11e9-96a8-0a1877e1c33d/
查看日志,已经不在报错
[root@iZbp1c4tqwd9kaykklejmxZ ~]# journalctl -u kubelet.service -f
-- Logs begin at Fri 2019-10-04 11:50:57 CST. --
Nov 01 13:45:30 iZbp1c4tqwd9kaykklejmxZ kubelet[9731]: W1101 13:45:30.552572 9731 reflector.go:270] object-"kube-system"/"kube-proxy-worker": watch of *v1.ConfigMap ended with: too old resource version: 2345126148 (2345131256)
Nov 01 13:46:36 iZbp1c4tqwd9kaykklejmxZ kubelet[9731]: W1101 13:46:36.578759 9731 plugin-defaults.go:32] flexVolume driver alicloud/nas: using default GetVolumeName for volume webquotes2-extreme-pv
See also