网站建设技术分类,成都网站开发哪个好,临沂网站建设熊掌号,打开网站搜索前言
Elasticsearch 是一个分布式、RESTful 风格的搜索和数据分析引擎#xff0c;能够解决不断涌现出的各种用例。 作为 Elastic Stack 的核心#xff0c;它集中存储您的数据#xff0c;帮助您发现意料之中以及意料之外的情况。
环境准备
软件 …前言
Elasticsearch 是一个分布式、RESTful 风格的搜索和数据分析引擎能够解决不断涌现出的各种用例。 作为 Elastic Stack 的核心它集中存储您的数据帮助您发现意料之中以及意料之外的情况。
环境准备
软件 OS系统 IP 节点 Elasticsearch7.7.0 centos 7 10.16.101.27 node-1 Elasticsearch7.7.0 centos 7 10.16.101.28 node-2 Elasticsearch7.7.0 centos 7 10.16.101.29 node-3
1.JDK环境安装以及配置
Elasticsearch7.7.0需要jdk11的允许环境所有我们安装Jdk11
# 已经JDK保存在OSS所有从OSS下载
cd root
wget https://starshop.obs.cn-south-1.myhuaweicloud.com/other/jdk-11.0.14_linux-x64_bin.tar.gz# 解压 jdk-11.0.14_linux-x64_bin.tar.gz 移动到/usr/java/jdk11目录下
tar xf jdk-11.0.14_linux-x64_bin.tar.gz
mv jdk-11.0.14_linux-x64_bin /usr/java/jdk11# 赋予java 执行权限
chmod 777 /usr/java/jdk11/bin/java
chmod 777 /usr/java/jdk11/jre/bin/java配置jdk环境变量
配置环境变量
vi /etc/profile
在文本末添加以下内容
---------------------------------------------------------------
#java环境变量
export JAVA_HOME/usr/java/jdk1.8
export PATH$JAVA_HOME/bin:$PATH
export CLASSPATH.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
----------------------------------------------------------------
执行一下命令 立即生效
source /etc/profile查看java版本
java -version2.开端口
# Elasticsearch7.7.0 需要开启9200 9300 9100端口进行通信
firewall-cmd --zonepublic --add-port9100/tcp --permanent firewall-cmd --zonepublic --add-port9200/tcp --permanent firewall-cmd --zonepublic --add-port9300/tcp --permanent firewall-cmd --reload
3.创建新用户用户启动Elasticsearch
# 因为Elasticsearch不能使用root用户启动所有必须新建用户
# 新建用户
adduser es
# 给新用户添加密码
passwd es安装 Elasticsearch7.7.0
https://www.elastic.co/cn/downloads/past-releases#elasticsearch
# 下载es
cd /root
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.7.0-linux-x86_64.tar.gz
# 解压文件到,移动到/usr/local/elastic/elasticsearch7
tar xf elasticsearch-7.7.0-linux-x86_64.tar.gz
mv elasticsearch-7.7.0-linux-x86_64 /usr/local/elastic/elasticsearch7
# 将elasticsearch7目录及路径下所有文件所有者均设为es用户
chown es:es -R /usr/local/elastic/elasticsearch71.修改系统配置文件
# 打开sysctl.conf文件。Linux默认不允许任何用户和应用程序直接开辟这么大的虚拟内存
vi /etc/sysctl.conf
# 文件sysctl.conf末尾加入
vm.max_map_count655360
# 保存退出后为了让系统控制权限配置生效
sysctl -p2.修改limits.conf文件
vi /etc/security/limits.conf
# 在文件内结束符前加上下面给出的几行代码,保存即可
* soft nofile 65536
* hard nofile 131072
* soft nproc 4096
* hard nproc 4096
* soft memlock unlimited
* hard memlock unlimited3.修改elasticsearch.yml(在27,28,29 编辑elasticsearch.yml)
# 切换到es用户
su es
# 编辑elasticsearch.yml
cd /usr/local/elastic/elasticsearch7/config
vi /usr/local/elastic/elasticsearch7/config/elasticsearch.yml
########################下面是文件内容########################################
# 集群名称同一集群下每台服务器的名称要一致否则会报错
cluster.name: my-elasticsearch
# 节点名称集群下每台机的节点名称唯一举例的机器节点名称设为node-1其他分别为node-2node-3
node.name: node-1
# ES数据储存路径保证该目录存在不存在则报错
path.data: /usr/local/elastic/elasticsearch7/data
# ES运行日志文件路径保证该目录存在不存在则报错
path.logs: /usr/local/elastic/elasticsearch7/logs
# 需求锁住物理内存避免操作系统的内存Swaping
bootstrap.memory_lock: true
# 外界访问ES ip地址(设置0.0.0.0表示可以通过该机器的任何ip访问)
network.host: 0.0.0.0
# 加入此配置可以解决集群的警告
network.publish_host: 10.16.101.27
# ES的访问端口号不设置则默认为9200
http.port: 9200
# 集群下所有机器的访问ES的url集合
discovery.seed_hosts: [10.16.101.27:9300, 10.16.101.28:9300, 10.16.101.29:9300]
# 集群下所有节点集合
cluster.initial_master_nodes: [node-1, node-2, node-3]
4 . 修改elasticsearch的VM配置
vi /usr/local/elastic/elasticsearch7/config/jvm.options修改VM为机器内存的一半
启动elasticsearch
1.设置elasticsearch开机启动
# 切换到root用户
su root
# 切换到/etc/rc.d/init.d目录
cd /etc/rc.d/init.d
# 编辑elasticsearch启动文件
vi elasticsearch
####################### 贴入下面的文本保存即可 ###############################
#!/bin/sh
#chkconfig: 2345 80 05
#description: elasticsearch 6.2.2
export JAVA_HOME/usr/java/jdk11
export ES_PATH/usr/local/elastic/elasticsearch7
case $1 in
start)su es!cd $ES_PATH./bin/elasticsearch -d -p pid
!echo elasticsearch startup;;
stop)es_pidps aux|grep elasticsearch | grep -v grep elasticsearch | awk {print $2}kill -9 $es_pidecho elasticsearch stopped;;
restart)es_pidps aux|grep elasticsearch | grep -v grep elasticsearch | awk {print $2}kill -9 $es_pidecho elasticsearch stoppedsu es!cd $ES_PATH./bin/elasticsearch -d -p pid
!echo elasticsearch startup;;
*)echo start|stop|restart;;
esac
exit $?# 刚编辑的文件需要赋予权限
chmod -X elasticsearch
# 添加到系统服务
chkconfig --add elasticsearch# 开启服务
service elasticsearch start
# 停止服务
service elasticsearch stop
# 重启服务
service elasticsearch restart
# 设置开机启动
chkconfig elasticsearch on在浏览器中输入 http://10.16.101.27:9200 代表安装成功查看集群是否成功 在浏览器输出入 http://10.16.101.27:9200/_cat/health?v 显示3节点代表成功
中文分词和拼音分词器安装
中文ik下载:https://github.com/medcl/elasticsearch-analysis-ik/releases 拼音分词器下载:https://github.com/medcl/elasticsearch-analysis-pinyin/releases
根据elasticsearch版本下载对应的分词器版本,必须版本对应
# 文件下载在root目录下
cd /root
#下载ik
wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.7.0/elasticsearch-analysis-ik-7.7.0.zip
# 下载pinyin
wget https://github.com/medcl/elasticsearch-analysis-pinyin/releases/download/v7.7.0/elasticsearch-analysis-pinyin-7.7.0.zip# 在elasticsearch 的plugins目录下新疆ik和pinyin文件夹
mkdir /usr/local/elastic/elasticsearch7/plugins/ik
mkdir /usr/local/elastic/elasticsearch7/plugins/pinyin
# unzip 将上面的2个zip直接解压到对应ik和pinyin目录下
unzip elasticsearch-analysis-ik-7.7.0.zip -d /usr/local/elastic/elasticsearch7/plugins/ik
unzip elasticsearch-analysis-pinyin-7.7.0.zip -d /usr/local/elastic/elasticsearch7/plugins/pinyin
# 赋予权限重启elasticsearch
chown es:es -R /usr/local/elastic/elasticsearch7
service elasticsearch restart使用crul命令输入下面的URL地址验证分词器是否成功
curl -X GET -H Content-Type: application/json http://localhost:9200/_analyze?prettytrue -d{text:中华五千年华夏};
若elasticsearch无法关闭可使用kill -9 pid 强制关闭在重启
# 查看pid
ps aux|grep elasticsearch
# 强制关闭
kill -9 xx
# 启动elasticsearch
service elasticsearch start安装浏览器ealsticsearch-head插件
ealsticsearch只是后端提供各种api那么怎么直观的使用它呢elasticsearch-head将是一款专门针对于elasticsearch的客户端工具 elasticsearch-head 需要nodejs的支持
1.下载安装nodejs
# 文件下载在root下
cd /root
# 下载最新稳定版nodejs
wget https://nodejs.org/dist/v16.15.1/node-v16.15.1-linux-x64.tar.xz
# 解压文件
tar xf node-v16.15.1-linux-x64.tar.xz
# 移动解压后目录到软件目录,并将目录重命名nodejs
mv node-v16.15.1-linux-x64 /usr/local/nodejs
# 查看node的版本
/usr/local/nodejs/bin/node -v
# 配置环境变量
vim /etc/profile#################################将下面文本贴入profile文件中保存###################
# 添加node_home变量
export NODE_HOME/usr/local/nodejs
# 把node_home变量添加到环境变量中
export PATH$PATH:$NODE_HOME/bin# 重新加载一下配置文件使新的配置生效
source /etc/profile
# 查看nodejs版本
node -version至此centos7上的nodejs运行环境安装成功下面我们安装elasticsearch-head
2.安装grunt构建工具
可以进行打包压缩、测试、执行等等的工作head插件就是通过grunt启动的。因此需要安装grunt
# 切换到nodejs安装目录
cd /usr/local/nodejs
# 配置阿里镜像
npm config set registry https://registry.npm.taobao.org
# 安装grunt
npm install -g grunt-cli
# 查看是否安装成功
grunt -version2.下载安装elasticsearch-head插件 GitHub - mobz/elasticsearch-head: A web front end for an elastic search cluster下载elasticsearch-head-master.zip将文件拷贝到/root目录下
cd /root
# 解压elasticsearch-head-master
unzip elasticsearch-head-master.zip
# 将文件移动到/usr/local/elastic/elasticsearch-head目录
mv elasticsearch-head-master /usr/local/elastic/elasticsearch-head
# 切换到/usr/local/elastic/elasticsearch-head目录下
cd /usr/local/elastic/elasticsearch-head
# 编译elasticsearch-head
npm install
# 运行 elasticsearch-head 后台运行
nohup npm run start 浏览器中输入上面的地址http://localhost:9100如图说明head插件已经安装成功了。 上图不能显示集群健康值,需要在配置一下elasticsearch.yml
cd /usr/local/elastic/elasticsearch7/config
vi elasticsearch.yml
# 在elasticsearch.yml文件中加入下面配置,保存即可
http.cors.enabled: true
http.cors.allow-origin: *
# 重启 elasticsearch
service elasticsearch restart3.ealsticsearch-head 开机启动
上面的配置ealsticsearch-head Xshell客户端关闭后导致9100端口也关闭所有必须配置开机自启动
cd /etc/init.d
# 新建elasticsearch-head启动脚本
vi elasticsearch-head
################## 写入下面的文本,根据自己时间安装情况配置#####################
#!/bin/sh
#chkconfig: 2345 80 05
#description: elasticsearch-head
# nodejs 安装的路径
export NODE_PATH/usr/local/nodejs
export PATH$PATH:$NODE_PATH/bin
# elasticsearch-head 的路径
cd /usr/local/elastic/elasticsearch-head
nohup npm run start /usr/local/elastic/elasticsearch-head/nohup.out 21
# 赋予 elasticsearch-head 权限
chmod x elasticsearch-head
# 设置开机执行sh
chkconfig --add elasticsearch-head
# 也可手动启动
service elasticsearch-head start启用x-pack验证
第一种
切换到es的用户下使用下面命令生成证书
bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass
启动elasticsearch
./elasticsearch -d
自动生成默认用户和密码
bin/elasticsearch-setup-passwords auto
# 手动设置
# bin/elasticsearch-setup-passwords interactive修改elastic超级用户密码
# 假设elastic 默认密码为 xxxxxxx
# curl -XPUT -u elastic:xxxxxxx http://localhost:9200/_xpack/security/user/elastic/_password -d { password : your_passwd }
浏览器验证 第二种
启动 Elasticsearch 程序
[elasticconsole bin]$ ./elasticsearch -dfuture versions of Elasticsearch will require Java 11; your Java version from [/usr/java/jdk1.8.0_181/jre] does not meet this requirement
创建密码
[elasticconsole bin]$ ./elasticsearch-setup-passwords interactive
future versions of Elasticsearch will require Java 11; your Java version from [/usr/java/jdk1.8.0_181/jre] does not meet this requirementUnexpected response code [500] from calling GET http://192.168.108.126:9200/_security/_authenticate?pretty
It doesnt look like the X-Pack security feature is enabled on this Elasticsearch node.
Please check if you have enabled X-Pack security in your elasticsearch.yml configuration file.ERROR: X-Pack Security is disabled by configuration.
需要设置 X-Pack
[elasticconsole bin]$ vim ../config/elasticsearch.ymlhttp.cors.enabled: truehttp.cors.allow-origin: *http.cors.allow-headers: Authorizationxpack.security.enabled: truexpack.security.transport.ssl.enabled: true
添加密码
[elasticconsole bin]$ ./elasticsearch-setup-passwords interactive
future versions of Elasticsearch will require Java 11; your Java version from [/usr/java/jdk1.8.0_181/jre] does not meet this requirement
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]yEnter password for [elastic]:
Reenter password for [elastic]:
Passwords do not match.
Try again.
Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]修改kibana
[rootconsole bin]# vim ../config/kibana.ymlelasticsearch.username: elasticelasticsearch.password: passwd修改密码
POST /_security/user/elastic/_password
{password: 123456
}修改密码之后需要重新设置kibana的配置文件才可以重新使用kibana
java 连接认证
package elasticSearch.highLevelClient;import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.XPackClient;import java.io.IOException;/*** author [tu.tengfei]* description* date 2019/8/10*/
public class ESHighClient {public static RestHighLevelClient client;public static void getESClient(){// client new RestHighLevelClient(RestClient.builder(
// new HttpHost(slave01, 9200, http),
// new HttpHost(master01, 9200, http)
// ));//需要用户名和密码的认证final CredentialsProvider credentialsProvider new BasicCredentialsProvider();credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(elastic, passwd));RestClientBuilder restClientBuilder RestClient.builder(new HttpHost(192.168.108.126, 9200, http)).setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {Overridepublic HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpAsyncClientBuilder) {return httpAsyncClientBuilder.setDefaultCredentialsProvider(credentialsProvider);}});client new RestHighLevelClient(restClientBuilder);}public static void clientClose(){if (client!null){try {client.close();} catch (IOException e) {e.printStackTrace();}}}
} Kibana7.4用户管理
1.下载kibana以及修改相应配置
cd /opt
wget https://artifacts.elastic.co/downloads/kibana/kibana-7.4.2-linux-x86_64.tar.gz
tar -zxvf kibana-7.4.2-linux-x86_64.tar.gz2.修改配置
cd /opt/kibana-7.4.2-linux-x86_64/config vim kibana.yml
# 中文
i18n.locale: zh-CN
server.port: 15601
server.host: 0.0.0.0
elasticsearch.hosts: [http://192.168.18.126:19200]
elasticsearch.username: kibana_system
elasticsearch.password: xxxxx
kibana.index: .kibana
xpack.reporting.encryptionKey: a_random_string
xpack.security.encryptionKey: something_at_least_32_characters
# 是否开启安全策略
# xpack.security.enabled: false3.kibana用户管理
/opt/kibana-7.4.2-linux-x86_64/bin/kibana --allow-root
添加用户并赋予相应的权限
