ElastAlert钉钉告警

DevOps
ELK评论4,002字数 5899阅读19分39秒阅读模式

安装python3环境

[root@prod-es-3 ~]# yum -y install wget openssl openssl-devel gcc gcc-c++
[root@prod-es-3 ~]# wget -c https://www.python.org/ftp/python/3.6.9/Python-3.6.9.tgz
[root@prod-es-3 ~]# tar xf Python-3.6.9.tgz
[root@prod-es-3 ~]# cd Python-3.6.9/
[root@prod-es-3 Python-3.6.9]# ./configure --prefix=/usr/local/python --with-openssl
[root@prod-es-3 Python-3.6.9]# make && make install
[root@prod-es-3 Python-3.6.9]# mv /usr/bin/python /usr/bin/python_back
[root@prod-es-3 Python-3.6.9]# ln -s /usr/local/python/bin/python3 /usr/bin/python
[root@prod-es-3 Python-3.6.9]# ln -s /usr/local/python/bin/pip3 /usr/bin/pip
[root@prod-es-3 Python-3.6.9]# pip install --upgrade pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/a4/6d/6463d49a933f547439d6b5b98b46af8742cc03ae83543e4d7688c2420f8b/pip-21.3.1-py3-none-any.whl (1.7MB)
    100% |████████████████████████████████| 1.7MB 19.9MB/s 
Installing collected packages: pip
  Found existing installation: pip 18.1
    Uninstalling pip-18.1:
      Successfully uninstalled pip-18.1
Successfully installed pip-21.3.1
[root@prod-es-3 Python-3.6.9]# cp /usr/bin/yum /usr/bin/yumback
[root@prod-es-3 Python-3.6.9]# cp /usr/libexec/urlgrabber-ext-down /usr/libexec/urlgrabber-ext-downback
[root@prod-es-3 Python-3.6.9]# sed -i '1s/python/python2.7/g' /usr/bin/yum
[root@prod-es-3 Python-3.6.9]# sed -i '1s/python/python2.7/g' /usr/libexec/urlgrabber-ext-down
[root@prod-es-3 Python-3.6.9]# python -V
Python 3.6.9
[root@prod-es-3 Python-3.6.9]# pip -V
pip 21.3.1 from /usr/local/python/lib/python3.6/site-packages/pip (python 3.6)

安装elastalert

[root@prod-es-3 Python-3.6.9]# cd /data/
[root@prod-es-3 data]# wget https://github.com/Yelp/elastalert/archive/v0.2.4.tar.gz
[root@prod-es-3 data]# tar zxf v0.2.4.tar.gz
[root@prod-es-3 data]# cd elastalert-0.2.4
[root@prod-es-3 elastalert-0.2.4]# pip install elasticsearch==7.0.0
[root@prod-es-3 elastalert-0.2.4]# pip install -r requirements.txt
[root@prod-es-3 elastalert-0.2.4]# python setup.py install
[root@prod-es-3 elastalert-0.2.4]# ll /usr/local/python/bin/elastalert* 
-rwxr-xr-x 1 root root 392 Jul 12 09:49 /usr/local/python/bin/elastalert
-rwxr-xr-x 1 root root 418 Jul 12 09:49 /usr/local/python/bin/elastalert-create-index
-rwxr-xr-x 1 root root 426 Jul 12 09:49 /usr/local/python/bin/elastalert-rule-from-kibana
-rwxr-xr-x 1 root root 412 Jul 12 09:49 /usr/local/python/bin/elastalert-test-rule
[root@prod-es-3 elastalert-0.2.4]# ln -s /usr/local/python/bin/elastalert* /usr/bin

使用钉钉告警

[root@prod-es-2 elastalert-0.2.4]#  wget https://github.com/xuyaoqiang/elastalert-dingtalk-plugin/archive/master.zip
[root@prod-es-2 elastalert-0.2.4]# unzip master.zip
Archive:  master.zip
c102ae8b20244f058204dbee5ff7cadc740f1375
   creating: elastalert-dingtalk-plugin-master/
  inflating: elastalert-dingtalk-plugin-master/.gitignore  
  inflating: elastalert-dingtalk-plugin-master/README.md  
  inflating: elastalert-dingtalk-plugin-master/config.yaml  
   creating: elastalert-dingtalk-plugin-master/elastalert_modules/
 extracting: elastalert-dingtalk-plugin-master/elastalert_modules/__init__.py  
  inflating: elastalert-dingtalk-plugin-master/elastalert_modules/dingtalk_alert.py  
  inflating: elastalert-dingtalk-plugin-master/requirements.txt  
   creating: elastalert-dingtalk-plugin-master/rules/
  inflating: elastalert-dingtalk-plugin-master/rules/api_error.yaml  
  inflating: elastalert-dingtalk-plugin-master/rules/avg_request_time.yaml  
[root@prod-es-2 elastalert-0.2.4]# cd elastalert-dingtalk-plugin-master
[root@prod-es-2 elastalert-dingtalk-plugin-master]# ll
total 12
-rw-r--r-- 1 root root 1762 Sep 15  2017 config.yaml
drwxr-xr-x 2 root root   50 Sep 15  2017 elastalert_modules
-rw-r--r-- 1 root root  688 Sep 15  2017 README.md
-rw-r--r-- 1 root root   92 Sep 15  2017 requirements.txt
drwxr-xr-x 2 root root   57 Sep 15  2017 rules

配置elastalert读取es信息

[root@prod-es-2 elastalert-dingtalk-plugin-master]# vim config.yaml 
rules_folder: rules
run_every:
  minutes: 1
buffer_time:
  minutes: 15
es_host: 10.0.3.174
es_port: 9200
es_username: elastic
es_password: root2758!@@
writeback_index: elastalert_status
alert_time_limit:
  days: 2

创建告警所用索引

[root@prod-es-2 elastalert-dingtalk-plugin-master]# pwd
/data/elastalert-0.2.4/elastalert-dingtalk-plugin-master
[root@prod-es-2 elastalert-dingtalk-plugin-master]# elastalert-create-index
Elastic Version: 7.17.5
Reading Elastic 6 index mappings:
Reading index mapping 'es_mappings/6/silence.json'
Reading index mapping 'es_mappings/6/elastalert_status.json'
Reading index mapping 'es_mappings/6/elastalert.json'
Reading index mapping 'es_mappings/6/past_elastalert.json'
Reading index mapping 'es_mappings/6/elastalert_error.json'
New index elastalert_status created
Done!
  • elastalert-create-index会创建一个索引，ElastAlert 会把执行记录存放到这个索引中，默认情况下，索引名叫 elastalert_status。其中有4个_type，都有自己的@timestamp 字段，所以同样也可以用kibana来查看这个索引的日志记录情况。
  • elastalert-rule-from-kibana从Kibana3已保存的仪表盘中读取Filtering 设置，帮助生成config.yaml里的配置。不过注意，它只会读取 filtering，不包括queries。
  • elastalert-test-rule测试自定义配置中的rule设置。

执行elastalert-create-index命令在ES创建索引，这不是必须的步骤，但是强烈建议创建。因为对于审计和测试很有用，并且重启ES不影响计数和发送alert

添加报警规则

  • 复制文件到主目录
[root@prod-es-2 elastalert]# scp -r /data/elastalert-0.2.4/elastalert-dingtalk-plugin-master/elastalert_modules /data/elastalert-0.2.4/
[root@prod-es-2 elastalert]# scp -r /data/elastalert-0.2.4/elastalert-dingtalk-plugin-master/config.yaml /data/elastalert-0.2.4/
[root@prod-es-2 elastalert]# scp -r /data/elastalert-0.2.4/elastalert-dingtalk-plugin-master/rules /data/elastalert-0.2.4/
  • 配置规则
[root@prod-es-2 rules]# pwd
/data/elastalert-0.2.4/rules
[root@prod-es-2 rules]# cat api_error.yaml 
name: logstash-logan错误信息
type: frequency
index: logstash-logan-*
num_events: 5
timeframe:
    minutes: 1
filter:
- query:
    query_string:
      query: "200"
    query_string:
      query: "error"
    
#只需要的字段 https://elastalert.readthedocs.io/en/latest/ruletypes.html#include
include: ["method", "url_path", "url_args", "status", "request_time","host","request","upstream","num_hits","message"]
alert:
- "elastalert_modules.dingtalk_alert.DingTalkAlerter"

dingtalk_webhook: "https://oapi.dingtalk.com/robot/send?access_token=93fa4a279ef8acae84a2526ef806c9391b3282c2e3f735229de30d77de61c7cd"
dingtalk_msgtype: "text"

手动启动看看有没有报错

[root@prod-es-2 elastalert-0.2.4]# python -m elastalert.elastalert --verbose --config /data/elastalert-0.2.4/config.yaml --rule /data/elastalert-0.2.4/rules/api_error.yaml

ElastAlert钉钉告警

继续阅读
ELK最后更新：2022-7-13
DevOps
  • 本文由 发表于 2022年7月12日 17:50:24
  • 除非特殊声明，本站文章均为原创，转载请务必保留本文链接
  • ElastAlert
EFK集群[案例] ELK

EFK集群[案例]

Elasticsearch集群配置信息 硬件配置信息 机器名/节点名 IP 内存 cpu 磁盘 us...
1,546评论
python定时清理ES 索引 ELK

python定时清理ES 索引

只保留三天 #!/usr/bin/env python3 # -*- coding:utf-8 -*- import os import datetime # 时间转化为字符串 n...
1,222评论
评论  0  访客  0

发表评论