目标

在生产环境中,我们需要除了监控kubernetes中的一些资源对象、节点以及组件,有时候还需要根据实际的业务添加自定义监控.添加自定义步骤如下: 1. 先创建一个ServiceMonitor对象,用于为Prometheus添加监控项 2. 将ServiceMonitor对象关联metrics数据接口的一个Service对象; 3. Service对象可以正确获取metrics数据

查看nginx完整的带metrics的文件

➜  manifests git:(master) ✗ cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: billy98/nginx-prometheus-metrics:latest
        ports:
        - name: http-metrics
          containerPort: 9527
        - name: web
          containerPort: 80
        - name: test
          containerPort: 1314
        imagePullPolicy: IfNotPresent
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx
  name: nginx
  namespace: default
spec:
  ports:
  - name: http-metrics
    port: 9527
    protocol: TCP
    targetPort: 9527
  - name: web
    port: 80
    protocol: TCP
    targetPort: 80
  - name: test
    port: 1314
    protocol: TCP
    targetPort: 1314
  selector:
    app: nginx
  type: NodePort                                                                        ➜  manifests git:(master) ✗

配置prometheus监控

➜  manifests git:(master) ✗ cat prometheus-serviceMonitornginx.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    app: nginx
  name: nginx
  namespace: monitoring
spec:
  endpoints:
  - interval: 15s
    port: http-metrics
  namespaceSelector:
    matchNames:
    - default
  selector:
    matchLabels:
      app: nginx
➜  manifests git:(master) ✗

创建nginx deployment


➜  manifests git:(master) ✗ k apply -f nginx.yaml
deployment.apps/nginx-demo created
service/nginx-demo unchanged
➜  manifests git:(master) ✗

创建nginx的prometheus监控

➜  manifests git:(master) ✗ k apply -f  prometheus-serviceMonitornginx.yaml
servicemonitor.monitoring.coreos.com/nginx-demo created
➜  manifests git:(master) ✗

查看映射的访问地址

➜  manifests git:(master) ✗ minikube service list
|-------------|-------------------------|--------------------------------|-----|
|  NAMESPACE  |          NAME           |          TARGET PORT           | URL |
|-------------|-------------------------|--------------------------------|-----|
| default     | kubernetes              | No node port                   |
| default     | nginx                   | http://192.168.64.2:31806      |
|             |                         | http://192.168.64.2:30190      |
|             |                         | http://192.168.64.2:31176      |
| kube-system | kube-controller-manager | No node port                   |
| kube-system | kube-dns                | No node port                   |
| kube-system | kube-scheduler          | No node port                   |
| kube-system | kubelet                 | No node port                   |
| monitoring  | alertmanager-main       | http://192.168.64.2:31175      |
| monitoring  | alertmanager-operated   | No node port                   |
| monitoring  | grafana                 | http://192.168.64.2:31263      |
| monitoring  | kube-state-metrics      | No node port                   |
| monitoring  | node-exporter           | No node port                   |
| monitoring  | prometheus-adapter      | No node port                   |
| monitoring  | prometheus-k8s          | http://192.168.64.2:30525      |
| monitoring  | prometheus-operated     | No node port                   |
| monitoring  | prometheus-operator     | No node port                   |
|-------------|-------------------------|--------------------------------|-----|
➜  manifests git:(master) ✗ k get svc
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                                      AGE
kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP                                      3d6h
nginx        NodePort    10.96.30.239   <none>        9527:31806/TCP,80:30190/TCP,1314:31176/TCP   17m
➜  manifests git:(master) ✗

访问映射的地址

http://192.168.64.2:31806/metrics

查看alerts是否创建成功

prometheus

prometheus “kubernetes”

prometheus学习系列(五)

prometheus Operator自定义监控nginx