Alert Manager 部署

1. 说明

此项目使用镜像 docker.io/prom/alertmanager:v0.24.0 ，离线环境请在部署前将其导入至 containerd 或私有镜像源地址中。

AlertManager 配置放在 configmap/conf-alert-manager 中。

2. 创建配置

使用以下配置创建 AlertManager 需要的相关的配置文件.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
apiVersion: v1
kind: ConfigMap
metadata:
  name: conf-alert-manager
  namespace: monitor-app
  labels:
    app: alert-manager
data:
  alertmanager.yml: |
    global:
      resolve_timeout: 30s
    route:
      group_by: [ 'instance' ]
      group_wait: 30s
      group_interval: 2m
      repeat_interval: 6h
      receiver: 'webhook'
    receivers:
      - name: 'webhook'
        webhook_configs:
          - url: 'http://172.16.63.51:8000/power-monitor/receive/receiveAlarmManagerAlarm.shtml'
    inhibit_rules:
      - source_matchers: [ severity="disaster" ]
        target_matchers: [ severity=~"normal|critical" ]
        equal: [ instanceIp ]
      - source_matchers: [ severity="critical" ]
        target_matchers: [ severity="normal" ]
        equal: [ instanceIp,project ]
      - source_matchers: [ priority="4" ]
        target_matchers: [ priority=~"3|2|1" ]
        equal: [ instance ]
      - source_matchers: [ priority="3" ]
        target_matchers: [ priority=~"2|1" ]
        equal: [ instance ]
      - source_matchers: [ priority="2" ]
        target_matchers: [ priority="1" ]
        equal: [ instance ]

3. 导入启动配置

使用以下配置创建 AlertManager 启动配置。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: sts-alert-manager
  namespace: monitor-app
  labels:
    app: alert-manager
spec:
  serviceName: svc-alert-manager
  selector:
    matchLabels:
      app: alert-manager
  replicas: 1
  template:
    metadata:
      labels:
        app: alert-manager
    spec:
      containers:
        - name: alert-manager
          image: docker.io/prom/alertmanager:v0.24.0
          args:
            - "--config.file=/etc/alertmanager/alertmanager.yml"
          volumeMounts:
            - name: alert-manager-conf
              mountPath: /etc/alertmanager/
      volumes:
        - name: alert-manager-conf
          configMap:
            name: conf-alert-manager

4. 创建 Service

使用以下配置，创建对应的 Service 配置

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
apiVersion: v1
kind: Service
metadata:
  labels:
    app: alert-manager
  name: svc-alert-manager
  namespace: monitor-app
spec:
  ports:
    - port: 9093
      name: alert-manager
  selector:
    app: alert-manager

5. 创建 Ingress

Service 创建完成后，使用以下配置将对应的 Service 暴露到 Ingress 中。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-alert-manager
  namespace: monitor-app
  labels:
    app: alert-manager
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
spec:
  ingressClassName: nginx-private
  tls:
    - hosts:
        - alert-manager.internal.d7z.net
      secretName: tls-pri-d7z
  rules:
    - host: alert-manager.internal.d7z.net
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: svc-alert-manager
                port:
                  name: alert-manager

6. 测试

配置导入完成后，使用如下命令查看结果：

1
kubectl get pods,svc,ingress,configmaps -n monitor-app -l app=alert-manager

待一切完成后，访问 https://alert-manager.internal.d7z.net 查看 web 页面。