EFK7.17.5集群

DevOps ELK评论1,257字数 23133阅读77分6秒阅读模式

系统优化

EFK7.17.5集群

配置ulimit

临时配置

#临时设置打开的最大文件数
ulimit -n 65536
#临时设置打开的最大进程数
ulimit -u 65536

永久配置

  • 大于或等于65536即可
vim /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
* soft nproc 65536
* hard nproc 65536

配置vm.max_map_count

[root@ip-10-0-3-174 ~]# vim /etc/sysctl.conf
fs.file-max=655360
vm.max_map_count=655360
[root@ip-10-0-3-174 ~]# sysctl -p
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_nonlocal_bind = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
net.ipv4.tcp_mem = 786432 1048576 1572864
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 16384 4194304
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_sack = 1
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.netdev_max_backlog = 262144
net.core.somaxconn = 20480
net.core.optmem_max = 81920
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_syn_retries = 3
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_max_tw_buckets = 20000
net.ipv4.tcp_max_orphans = 3276800
net.ipv4.tcp_timestamps = 1 #?
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.ip_local_port_range = 10001 65000
vm.overcommit_memory = 0
vm.swappiness = 10
fs.file-max = 655360
vm.max_map_count = 655360

Elasticsearch7.17.5集群

下载es

[centos@ip-10-0-3-195 ~]$ cd /data/
[centos@ip-10-0-3-195 data]$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.5-linux-x86_64.tar.gz
[centos@ip-10-0-3-53 ~]$ cd /data/
[centos@ip-10-0-3-53 data]$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.5-linux-x86_64.tar.gz
[centos@ip-10-0-3-174 ~]$ cd /data/
[centos@ip-10-0-3-174 data]$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.5-linux-x86_64.tar.gz
[centos@ip-10-0-3-195 data]$ tar zxf elasticsearch-7.17.5-linux-x86_64.tar.gz 
[centos@ip-10-0-3-53 data]$ tar zxf elasticsearch-7.17.5-linux-x86_64.tar.gz 
[centos@ip-10-0-3-174 data]$ tar zxf elasticsearch-7.17.5-linux-x86_64.tar.gz

修改es配置

  • 第一个节点
[centos@prod-es-1 config]$ pwd
/data/elasticsearch-7.17.5/config
[centos@prod-es-1 config]$ ll
total 40
-rw-rw---- 1 centos centos  1042 Jun 23 21:55 elasticsearch-plugins.example.yml
-rw-rw---- 1 centos centos  3415 Jun 23 21:55 elasticsearch.yml
-rw-rw---- 1 centos centos  3257 Jun 23 21:55 jvm.options
drwxr-x--- 2 centos centos     6 Jun 23 21:59 jvm.options.d
-rw-rw---- 1 centos centos 19304 Jun 23 21:58 log4j2.properties
-rw-rw---- 1 centos centos   473 Jun 23 21:58 role_mapping.yml
-rw-rw---- 1 centos centos   197 Jun 23 21:58 roles.yml
-rw-rw---- 1 centos centos     0 Jun 23 21:58 users
-rw-rw---- 1 centos centos     0 Jun 23 21:58 users_roles
[centos@prod-es-1 config]$ cp elasticsearch.yml elasticsearch.ymlback
[centos@prod-es-1 config]$ > elasticsearch.yml
[centos@prod-es-1 config]$ vim elasticsearch.yml
cluster.name: lingoace-es
node.name: prod-es-1
node.master: true
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 10.0.3.195
http.port: 9200
transport.tcp.port: 9300
transport.tcp.compress: true
discovery.seed_hosts: ["10.0.3.195", "10.0.3.53","10.0.3.174"]
cluster.initial_master_nodes: ["10.0.3.195", "10.0.3.53"]
discovery.zen.minimum_master_nodes: 2
cluster.fault_detection.leader_check.interval: 15s
discovery.cluster_formation_warning_timeout: 30s
cluster.join.timeout: 30s
cluster.publish.timeout: 90s
cluster.routing.allocation.cluster_concurrent_rebalance: 16
cluster.routing.allocation.node_concurrent_recoveries: 16
cluster.routing.allocation.node_initial_primaries_recoveries: 16
http.cors.enabled: true
http.cors.allow-origin: "*"
  • 第二个节点
[centos@prod-es-2 config]$ cd /data/elasticsearch-7.17.5/config
[centos@prod-es-2 config]$ ll
total 40
-rw-rw---- 1 centos centos  1042 Jun 23 21:55 elasticsearch-plugins.example.yml
-rw-rw---- 1 centos centos  3415 Jun 23 21:55 elasticsearch.yml
-rw-rw---- 1 centos centos  3257 Jun 23 21:55 jvm.options
drwxr-x--- 2 centos centos     6 Jun 23 21:59 jvm.options.d
-rw-rw---- 1 centos centos 19304 Jun 23 21:58 log4j2.properties
-rw-rw---- 1 centos centos   473 Jun 23 21:58 role_mapping.yml
-rw-rw---- 1 centos centos   197 Jun 23 21:58 roles.yml
-rw-rw---- 1 centos centos     0 Jun 23 21:58 users
-rw-rw---- 1 centos centos     0 Jun 23 21:58 users_roles
[centos@prod-es-2 config]$ cp elasticsearch.yml elasticsearch.ymlback
[centos@prod-es-2 config]$ > elasticsearch.yml
[centos@prod-es-2 config]$ vim elasticsearch.yml
cluster.name: lingoace-es
node.name: prod-es-2
node.master: true
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 10.0.3.53
http.port: 9200
transport.tcp.port: 9300
transport.tcp.compress: true
discovery.seed_hosts: ["10.0.3.195", "10.0.3.53","10.0.3.174"]
cluster.initial_master_nodes: ["10.0.3.53", "10.0.3.195"]
discovery.zen.minimum_master_nodes: 2
cluster.fault_detection.leader_check.interval: 15s
discovery.cluster_formation_warning_timeout: 30s
cluster.join.timeout: 30s
cluster.publish.timeout: 90s
cluster.routing.allocation.cluster_concurrent_rebalance: 16
cluster.routing.allocation.node_concurrent_recoveries: 16
cluster.routing.allocation.node_initial_primaries_recoveries: 16
http.cors.enabled: true
http.cors.allow-origin: "*"
  • 第三个节点
[centos@prod-es-3 config]$ cd /data/elasticsearch-7.17.5/config
[centos@prod-es-3 config]$ cp elasticsearch.yml elasticsearch.ymlback
[centos@prod-es-3 config]$ ll
total 44
-rw-rw---- 1 centos centos  1042 Jun 23 21:55 elasticsearch-plugins.example.yml
-rw-rw---- 1 centos centos  3415 Jun 23 21:55 elasticsearch.yml
-rw-rw---- 1 centos centos  3415 Jul 11 10:44 elasticsearch.ymlback
-rw-rw---- 1 centos centos  3257 Jun 23 21:55 jvm.options
drwxr-x--- 2 centos centos     6 Jun 23 21:59 jvm.options.d
-rw-rw---- 1 centos centos 19304 Jun 23 21:58 log4j2.properties
-rw-rw---- 1 centos centos   473 Jun 23 21:58 role_mapping.yml
-rw-rw---- 1 centos centos   197 Jun 23 21:58 roles.yml
-rw-rw---- 1 centos centos     0 Jun 23 21:58 users
-rw-rw---- 1 centos centos     0 Jun 23 21:58 users_roles
[centos@prod-es-3 config]$ > elasticsearch.yml
cluster.name: lingoace-es
node.name: prod-es-3
node.master: true
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 10.0.3.174
http.port: 9200
transport.tcp.port: 9300
transport.tcp.compress: true
discovery.seed_hosts: ["10.0.3.195", "10.0.3.53","10.0.3.174"]
cluster.initial_master_nodes: ["10.0.3.174", "10.0.3.53"]
discovery.zen.minimum_master_nodes: 2
cluster.fault_detection.leader_check.interval: 15s
discovery.cluster_formation_warning_timeout: 30s
cluster.join.timeout: 30s
cluster.publish.timeout: 90s
cluster.routing.allocation.cluster_concurrent_rebalance: 16
cluster.routing.allocation.node_concurrent_recoveries: 16
cluster.routing.allocation.node_initial_primaries_recoveries: 16
http.cors.enabled: true
http.cors.allow-origin: "*"

创建存储目录

[centos@prod-es-1 config]$ mkdir -p /data/elasticsearch/data
[centos@prod-es-1 config]$ mkdir -p /data/elasticsearch/logs
[centos@prod-es-2 config]$ mkdir -p /data/elasticsearch/data
[centos@prod-es-2 config]$ mkdir -p /data/elasticsearch/logs
[centos@prod-es-3 config]$ mkdir -p /data/elasticsearch/data
[centos@prod-es-3 config]$ mkdir -p /data/elasticsearch/logs

修改内存

  • 总内存的一半
[centos@prod-es-1 config]$ vim /data/elasticsearch-7.17.5/config/jvm.options
-Xms4g
-Xmx4g
[centos@prod-es-2 config]$ vim /data/elasticsearch-7.17.5/config/jvm.options
-Xms4g
-Xmx4g
[centos@prod-es-3 config]$ vim /data/elasticsearch-7.17.5/config/jvm.options
-Xms4g
-Xmx4g

配置证书

TLS需要X.509证书才能对与之通信的应用程序执行加密和身份验证。为了使节点之间的通信真正安全,必须对证书进行验证。在Elasticsearch集群中验证证书真实性的推荐方法是信任签署证书的证书颁发机构(CA)。这样,将节点添加到群集时,它们只需要使用由同一CA签名的证书,即可自动允许该节点加入群集。

生成节点证书

命令 elasticsearch-certutil 简化了生成证书的过程,它负责生成CA并与CA签署证书。

a、创建证书颁发机构CA
随便进入一个节点的bin 目录下执行elasticsearch-certutil 命令即可。

执行这个命令之后:

  • 会让你输入生成elastic-stack-ca.p12文件放在哪。(直接回车,放在当前目录)
  • 回车之后让你输入密码,该密码是让你保护文件和密钥的。如果你以后还要加集群的话,要记得输入的密码。
[centos@prod-es-3 elasticsearch-7.17.5]$ pwd
/data/elasticsearch-7.17.5
[centos@prod-es-3 elasticsearch-7.17.5]$ bin/elasticsearch-certutil ca
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.

Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authority

By default the 'ca' mode produces a single PKCS#12 output file which holds:
    * The CA certificate
    * The CA's private key

If you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private key

Please enter the desired output file [elastic-stack-ca.p12]: 
Enter password for elastic-stack-ca.p12 : 
[centos@prod-es-3 elasticsearch-7.17.5]$ ll
total 652
drwxr-xr-x  2 centos centos   4096 Jun 23 22:02 bin
drwxr-xr-x  3 centos centos    239 Jul 11 10:58 config
-rw-------  1 centos centos   2527 Jul 11 13:25 elastic-stack-ca.p12
drwxr-xr-x  8 centos centos     96 Jun 23 22:02 jdk
drwxr-xr-x  3 centos centos   4096 Jun 23 22:02 lib
-rw-r--r--  1 centos centos   3860 Jun 23 21:55 LICENSE.txt
drwxr-xr-x  2 centos centos      6 Jun 23 21:59 logs
drwxr-xr-x 61 centos centos   4096 Jun 23 22:03 modules
-rw-r--r--  1 centos centos 640930 Jun 23 21:59 NOTICE.txt
drwxr-xr-x  2 centos centos      6 Jun 23 21:59 plugins
-rw-r--r--  1 centos centos   2710 Jun 23 21:55 README.asciidoc

b、生成证书和私钥

执行命令之后需要你操作3次:

  • 第一次,输入上面生成CA的密码,没有设置直接回车
  • 第二次,生成的文件路径,直接回车
  • 第三次,生成这次证书与私钥文件的密码,建议和上面生成CA一致(怕忘记密码,也可以直接回车)
[centos@prod-es-3 elasticsearch-7.17.5]$ pwd
/data/elasticsearch-7.17.5
[centos@prod-es-3 elasticsearch-7.17.5]$ bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'cert' mode generates X.509 certificate and private keys.
    * By default, this generates a single certificate and key for use
       on a single instance.
    * The '-multiple' option will prompt you to enter details for multiple
       instances and will generate a certificate and key for each one
    * The '-in' option allows for the certificate generation to be automated by describing
       the details of each instance in a YAML file

    * An instance is any piece of the Elastic Stack that requires an SSL certificate.
      Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
      may all require a certificate and private key.
    * The minimum required value for each instance is a name. This can simply be the
      hostname, which will be used as the Common Name of the certificate. A full
      distinguished name may also be used.
    * A filename value may be required for each instance. This is necessary when the
      name would result in an invalid file or directory name. The name provided here
      is used as the directory name (within the zip) and the prefix for the key and
      certificate files. The filename is required if you are prompted and the name
      is not displayed in the prompt.
    * IP addresses and DNS names are optional. Multiple values can be specified as a
      comma separated string. If no IP addresses or DNS names are provided, you may
      disable hostname verification in your SSL configuration.

    * All certificates generated by this tool will be signed by a certificate authority (CA)
      unless the --self-signed command line option is specified.
      The tool can automatically generate a new CA for you, or you can provide your own with
      the --ca or --ca-cert command line options.

By default the 'cert' mode produces a single PKCS#12 output file which holds:
    * The instance certificate
    * The private key for the instance certificate
    * The CA certificate

If you specify any of the following options:
    * -pem (PEM formatted output)
    * -keep-ca-key (retain generated CA key)
    * -multiple (generate multiple certificates)
    * -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key files

Enter password for CA (elastic-stack-ca.p12) : 
Please enter the desired output file [elastic-certificates.p12]: 
Enter password for elastic-certificates.p12 : 

Certificates written to /data/elasticsearch-7.17.5/elastic-certificates.p12

This file should be properly secured as it contains the private key for 
your instance.

This file is a self contained file and can be copied and used 'as is'
For each Elastic product that you wish to configure, you should copy
this '.p12' file to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.
[centos@prod-es-3 elasticsearch-7.17.5]$ ll
total 656
drwxr-xr-x 2 centos centos 4096 Jun 23 22:02 bin
drwxr-xr-x 3 centos centos 239 Jul 11 10:58 config
-rw------- 1 centos centos 3443 Jul 11 13:29 elastic-certificates.p12
-rw------- 1 centos centos 2527 Jul 11 13:25 elastic-stack-ca.p12
drwxr-xr-x 8 centos centos 96 Jun 23 22:02 jdk
drwxr-xr-x 3 centos centos 4096 Jun 23 22:02 lib
-rw-r--r-- 1 centos centos 3860 Jun 23 21:55 LICENSE.txt
drwxr-xr-x 2 centos centos 6 Jun 23 21:59 logs
drwxr-xr-x 61 centos centos 4096 Jun 23 22:03 modules
-rw-r--r-- 1 centos centos 640930 Jun 23 21:59 NOTICE.txt
drwxr-xr-x 2 centos centos 6 Jun 23 21:59 plugins
-rw-r--r-- 1 centos centos 2710 Jun 23 21:55 README.asciidoc

命令执行完之后会生成一个elastic-certificates.p12 文件,这个就是各节点通信的凭证

只需要一个节点生成凭证即可。

添加证书文件

把证书凭证复制到各个节点一份

[centos@prod-es-3 elasticsearch-7.17.5]$ mv elastic-* config/
[centos@prod-es-3 elasticsearch-7.17.5]$ ll config/
total 52
-rw------- 1 centos centos  3443 Jul 11 13:29 elastic-certificates.p12
-rw-rw---- 1 centos centos  1042 Jun 23 21:55 elasticsearch-plugins.example.yml
-rw-rw---- 1 centos centos   862 Jul 11 10:53 elasticsearch.yml
-rw-rw---- 1 centos centos  3415 Jul 11 10:44 elasticsearch.ymlback
-rw------- 1 centos centos  2527 Jul 11 13:25 elastic-stack-ca.p12
-rw-rw---- 1 centos centos  3251 Jul 11 10:58 jvm.options
drwxr-x--- 2 centos centos     6 Jun 23 21:59 jvm.options.d
-rw-rw---- 1 centos centos 19304 Jun 23 21:58 log4j2.properties
-rw-rw---- 1 centos centos   473 Jun 23 21:58 role_mapping.yml
-rw-rw---- 1 centos centos   197 Jun 23 21:58 roles.yml
-rw-rw---- 1 centos centos     0 Jun 23 21:58 users
-rw-rw---- 1 centos centos     0 Jun 23 21:58 users_roles
[centos@prod-es-2 config]$ pwd
/data/elasticsearch-7.17.5/config
[centos@prod-es-2 config]$ ll
total 52
-rw------- 1 centos centos 3443 Jul 11 13:29 elastic-certificates.p12
-rw-rw---- 1 centos centos 1042 Jun 23 21:55 elasticsearch-plugins.example.yml
-rw-rw---- 1 centos centos 862 Jul 11 10:51 elasticsearch.yml
-rw-rw---- 1 centos centos 3415 Jul 11 10:44 elasticsearch.ymlback
-rw------- 1 centos centos 2527 Jul 11 13:25 elastic-stack-ca.p12
-rw-rw---- 1 centos centos 3251 Jul 11 10:58 jvm.options
drwxr-x--- 2 centos centos 6 Jun 23 21:59 jvm.options.d
-rw-rw---- 1 centos centos 19304 Jun 23 21:58 log4j2.properties
-rw-rw---- 1 centos centos 473 Jun 23 21:58 role_mapping.yml
-rw-rw---- 1 centos centos 197 Jun 23 21:58 roles.yml
-rw-rw---- 1 centos centos 0 Jun 23 21:58 users
-rw-rw---- 1 centos centos 0 Jun 23 21:58 users_roles
[centos@prod-es-1 config]$ pwd
/data/elasticsearch-7.17.5/config
[centos@prod-es-1 config]$ ll
total 52
-rw------- 1 centos centos 3443 Jul 11 13:29 elastic-certificates.p12
-rw-rw---- 1 centos centos 1042 Jun 23 21:55 elasticsearch-plugins.example.yml
-rw-rw---- 1 centos centos 862 Jul 11 10:48 elasticsearch.yml
-rw-rw---- 1 centos centos 3415 Jul 11 10:44 elasticsearch.ymlback
-rw------- 1 centos centos 2527 Jul 11 13:25 elastic-stack-ca.p12
-rw-rw---- 1 centos centos 3251 Jul 11 10:57 jvm.options
drwxr-x--- 2 centos centos 6 Jun 23 21:59 jvm.options.d
-rw-rw---- 1 centos centos 19304 Jun 23 21:58 log4j2.properties
-rw-rw---- 1 centos centos 473 Jun 23 21:58 role_mapping.yml
-rw-rw---- 1 centos centos 197 Jun 23 21:58 roles.yml
-rw-rw---- 1 centos centos 0 Jun 23 21:58 users
-rw-rw---- 1 centos centos 0 Jun 23 21:58 users_roles

追加证书配置到es

[centos@prod-es-3 config]$ pwd
/data/elasticsearch-7.17.5/config
[centos@prod-es-3 config]$ grep -v "^$" elasticsearch.yml | grep -v "^#"
cluster.name: lingoace-es
node.name: prod-es-3
node.master: true
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 10.0.3.174
http.port: 9200
transport.tcp.port: 9300
transport.tcp.compress: true
discovery.seed_hosts: ["10.0.3.174", "10.0.3.53","10.0.3.195"]
cluster.initial_master_nodes: ["10.0.3.174", "10.0.3.53"]
discovery.zen.minimum_master_nodes: 2
cluster.fault_detection.leader_check.interval: 15s
discovery.cluster_formation_warning_timeout: 30s
cluster.join.timeout: 30s
cluster.publish.timeout: 90s
cluster.routing.allocation.cluster_concurrent_rebalance: 16
cluster.routing.allocation.node_concurrent_recoveries: 16
cluster.routing.allocation.node_initial_primaries_recoveries: 16
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: true
xpack.security.authc.accept_default_password: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /data/elasticsearch-7.17.5/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /data/elasticsearch-7.17.5/config/elastic-certificates.p12
[centos@prod-es-2 config]$ pwd
/data/elasticsearch-7.17.5/config
[centos@prod-es-2 config]$ grep -v "^$" elasticsearch.yml | grep -v "^#"
cluster.name: lingoace-es
node.name: prod-es-2
node.master: true
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 10.0.3.53
http.port: 9200
transport.tcp.port: 9300
transport.tcp.compress: true
discovery.seed_hosts: ["10.0.3.53", "10.0.3.195","10.0.3.174"]
cluster.initial_master_nodes: ["10.0.3.53", "10.0.3.195"]
discovery.zen.minimum_master_nodes: 2
cluster.fault_detection.leader_check.interval: 15s
discovery.cluster_formation_warning_timeout: 30s
cluster.join.timeout: 30s
cluster.publish.timeout: 90s
cluster.routing.allocation.cluster_concurrent_rebalance: 16
cluster.routing.allocation.node_concurrent_recoveries: 16
cluster.routing.allocation.node_initial_primaries_recoveries: 16
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: true
xpack.security.authc.accept_default_password: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /data/elasticsearch-7.17.5/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /data/elasticsearch-7.17.5/config/elastic-certificates.p12
[centos@prod-es-1 config]$ pwd
/data/elasticsearch-7.17.5/config
[centos@prod-es-1 config]$ grep -v "^$" elasticsearch.yml | grep -v "^#"
cluster.name: lingoace-es
node.name: prod-es-1
node.master: true
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 10.0.3.195
http.port: 9200
transport.tcp.port: 9300
transport.tcp.compress: true
discovery.seed_hosts: ["10.0.3.195", "10.0.3.53","10.0.3.174"]
cluster.initial_master_nodes: ["10.0.3.195", "10.0.3.174"]
discovery.zen.minimum_master_nodes: 2
cluster.fault_detection.leader_check.interval: 15s
discovery.cluster_formation_warning_timeout: 30s
cluster.join.timeout: 30s
cluster.publish.timeout: 90s
cluster.routing.allocation.cluster_concurrent_rebalance: 16
cluster.routing.allocation.node_concurrent_recoveries: 16
cluster.routing.allocation.node_initial_primaries_recoveries: 16
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: true
xpack.security.authc.accept_default_password: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /data/elasticsearch-7.17.5/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /data/elasticsearch-7.17.5/config/elastic-certificates.p12

要注意的是上面的path记得改成对应节点config下的elastic-certificates.p12

添加密码到密码库

  • 因为之前生成CA 和生成凭证都设置了密码,所以把密码添加到密钥库中。

每个节点都得添加

[centos@prod-es-3 elasticsearch-7.17.5]$ pwd
/data/elasticsearch-7.17.5
[centos@prod-es-3 elasticsearch-7.17.5]$ bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
The elasticsearch keystore does not exist. Do you want to create it? [y/N]y
Enter value for xpack.security.transport.ssl.keystore.secure_password: 
[centos@prod-es-3 elasticsearch-7.17.5]$ bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Enter value for xpack.security.transport.ssl.truststore.secure_password: 
[centos@prod-es-2 elasticsearch-7.17.5]$ pwd
/data/elasticsearch-7.17.5
[centos@prod-es-2 elasticsearch-7.17.5]$ bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
The elasticsearch keystore does not exist. Do you want to create it? [y/N]y
Enter value for xpack.security.transport.ssl.keystore.secure_password: 
[centos@prod-es-2 elasticsearch-7.17.5]$ bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Enter value for xpack.security.transport.ssl.truststore.secure_password: 
[centos@prod-es-1 elasticsearch-7.17.5]$ pwd
/data/elasticsearch-7.17.5
[centos@prod-es-1 elasticsearch-7.17.5]$ bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
The elasticsearch keystore does not exist. Do you want to create it? [y/N]y
Enter value for xpack.security.transport.ssl.keystore.secure_password: 
[centos@prod-es-1 elasticsearch-7.17.5]$ bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Enter value for xpack.security.transport.ssl.truststore.secure_password:

修改启动脚本为本机Java

  • 三台都修改一下
[centos@prod-es-3 bin]$ vim /data/elasticsearch-7.17.5/bin/elasticsearch-env
ES_JAVA_HOME="/data/app/jdk-13.0.1"

EFK7.17.5集群

启动服务

先前台启动,看看有没有错误

[centos@prod-es-1 bin]$ pwd
/data/elasticsearch-7.17.5/bin
[centos@prod-es-1 bin]$ ./elasticsearch
[centos@prod-es-2 config]$ cd /data/elasticsearch-7.17.5/bin
[centos@prod-es-2 bin]$ ./elasticsearch
[centos@prod-es-3 config]$ cd /data/elasticsearch-7.17.5/bin
[centos@prod-es-3 bin]$ ./elasticsearch

后台启动

[centos@prod-es-1 bin]$ ./elasticsearch -d

配置密码

elasticsearch-setup-passwords工具是首次设置内置用户密码的最简单方法。它使用elastic用户的引导程序密码来运行用户管理API请求
[centos@prod-es-3 elasticsearch-7.17.5]$ pwd
/data/elasticsearch-7.17.5

[centos@prod-es-3 elasticsearch-7.17.5]$ bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y

Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana_system]:
Reenter password for [kibana_system]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

安装kibana

[centos@prod-es-3 bin]$ cd /data/
[centos@prod-es-3 data]$ wget https://artifacts.elastic.co/downloads/kibana/kibana-7.17.5-linux-x86_64.tar.gz
[centos@prod-es-3 data]$ tar zxf kibana-7.17.5-linux-x86_64.tar.gz 
[centos@prod-es-3 data]$ cd kibana-7.17.5-linux-x86_64
[centos@prod-es-3 kibana-7.17.5-linux-x86_64]$ cd config/
[centos@prod-es-3 config]$ grep -v "^$" kibana.yml | grep -v "^#"
server.port: 5601
server.host: "10.0.3.174"
server.publicBaseUrl: "http://10.0.3.174:5601"
server.name: "kibana"
elasticsearch.hosts: ["http://10.0.3.53:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "root2758!@@"
i18n.locale: "zh-CN"
xpack.security.encryptionKey: Hz*9yFFaPejHvCkhT*ddNx%WsBgxVSCQ # 自己随意生成的32位加密key
[centos@prod-es-3 bin]$ pwd
/data/kibana-7.17.5-linux-x86_64/bin
[centos@prod-es-3 bin]$ nohup ./kibana &

登陆kibana

  • 账户和密码是elastic:root2758!@@,之前自己设置的密码

EFK7.17.5集群

继续阅读
ELK最后更新:2022-7-12
DevOps
  • 本文由 发表于 2022年7月11日 16:34:11
  • 除非特殊声明,本站文章均为原创,转载请务必保留本文链接
EFK集群[案例] ELK

EFK集群[案例]

Elasticsearch集群配置信息 硬件配置信息 机器名/节点名 IP 内存 cpu 磁盘 us...
python定时清理ES 索引 ELK

python定时清理ES 索引

只保留三天 #!/usr/bin/env python3 # -*- coding:utf-8 -*- import os import datetime # 时间转化为字符串 n...
评论  0  访客  0

发表评论