网站生成静态页面工具,树状结构wordpress模板,wordpress电商主题下载,广州网站建设总结这篇其实去年就写好了#xff0c;孰知就在12月31日那一天打进决赛圈#xff0c;一躺#xff0c;二过年#xff0c;三休假#xff0c;四加班#xff0c;居然到了三个月以后#xff0c;才有机会将它发出来…… 一年也就四个季度不是#xff0c;实在是光阴荏苒#xff0c… 这篇其实去年就写好了孰知就在12月31日那一天打进决赛圈一躺二过年三休假四加班居然到了三个月以后才有机会将它发出来…… 一年也就四个季度不是实在是光阴荏苒岁月蹉跎。 本来也打算给这个系列换个名字。从学习Centos开始结果写着写着已经完全不是Centos那些东西了。不过考虑到换名字也很麻烦不如保持初心继续Centos的好。 Click House是目前比较热的数据库选择这一节我们尝试搭建其使用环境并基于VSCODE搭建编程环境。ClickHouse是俄罗斯商业公司开发的一款分布式列数据库软件独立于Hadoop体系别树一格地提供了高性能的数据库服务支撑。Click House的官网是有详细的中文支持文档的可参考什么是ClickHouse | ClickHouse Docs 一、系统要求 根据官方的描述ClickHouse支持的CPU架构有x86_64、AArch64和PowerPC64LE操作系统包括Linux、FreeBSD和Mac OS X。之所以要提及CPU架构是因为ClickHouse在优化中使用了特定的CPU指令集。这也是网络安全工具的底层支撑平台常用的一种策略比如DPDK和HyperScan等都是如此。具体在Intel的x86_64架构上ClickHouse主要依靠SSE 4.2指令集来进行优化。 SSE指令集扩展于早期Pentium III的MMX指令集。MMX主要针对多媒体的大量整形计算需求提供了寄存器级别的并行计算能力可以借用浮点计算的寄存器空间提供一条指令计算多条数据SIMDSingle Instruction Multi-Data的能力。随着大数据的发展各种网络数据处理和大数据应用对此类运算加速的需求快速增长最终Intel推出了专门的SSE指令集从而不用争抢浮点计算的资源专门提供SIMD的计算资源。SSE实际也就是Internet Streaming SIMD Extensions的简称。 确认系统是否支持SSE指令集可以在系统之后从/proc/cpuinfo获取系统运行时的CPU相关信息的描述从中搜索“SSE4_2”字样 也可以使用官网上的检查方法。命令中表示前条命令执行成功后执行||表示前条命令执行不成功时执行-q 代表静默执行否则会把上图的执行结果也打印出来。
[rootpig sysconfig]# grep -q sse4_2 /proc/cpuinfo echo SSE 4.2 supported || echo SSE 4.2 not supported
SSE 4.2 supported
[rootpig sysconfig]#对于AArch64和PowerPC64LE架构因为不支持SSE指令集所以需要从ClickHouse的源代码进行编译的方式安装部署。 二、VMware上单点部署ClickHouse 按照Hadoop的上手流程我们首先在单节点上部署ClickHouse并在其上关联VSCODE构建编程环境然后再考虑集群部署的问题。 一添加安装源 官方对Redhat/CentOS和Debian/Ubuntu类的Linux均提供了安装源 1. Redhat/CentOS添加安装源 在/etc/yum.repo.d/目录下添加镜像源可以完全手工完成前面经常搞此处不赘述也可以基于yum-utils工具来“正规化”达成
[rootpig yum.repos.d]# yum install yum-utils
上次元数据过期检查1:24:34 前执行于 2022年12月19日 星期一 21时12分06秒。
软件包 yum-utils-4.0.21-11.el8.noarch 已安装。
依赖关系解决。
软件包 架构 版本 仓库 大小升级:dnf-plugins-core noarch 4.0.21-16.el8 baseos 75 kpython3-dnf-plugins-core noarch 4.0.21-16.el8 baseos 258 kyum-utils noarch 4.0.21-16.el8 baseos 74 k
事务概要升级 3 软件包
总下载407 k
确定吗[y/N] y
下载软件包
(1/3): yum-utils-4.0.21-16.el8.noarch.rpm 54 kB/s | 74 kB 00:01
(2/3): dnf-plugins-core-4.0.21-16.el8.noarch.rpm 44 kB/s | 75 kB 00:01
(3/3): python3-dnf-plugins-core-4.0.21-16.el8.noarch.rpm 60 kB/s | 258 kB 00:04
--------------------------------------------------------------------------------------------------------------------------------------------
总计 84 kB/s | 407 kB 00:04
CentOS Stream 8 - BaseOS 1.6 MB/s | 1.6 kB 00:00
导入 GPG 公钥 0x8483C65D:Userid: CentOS (CentOS Official Signing Key) securitycentos.org指纹: 99DB 70FA E1D7 CE22 7FB6 4882 05B5 55B3 8483 C65D来自: /etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
确定吗[y/N] y
导入公钥成功
运行事务检查
事务检查成功。
运行事务测试
事务测试成功。
运行事务准备中 : 1/1 升级 : python3-dnf-plugins-core-4.0.21-16.el8.noarch 1/6
……………………运行脚本: python3-dnf-plugins-core-4.0.21-11.el8.noarch 6/6 验证 : dnf-plugins-core-4.0.21-16.el8.noarch 1/6
………………验证 : yum-utils-4.0.21-11.el8.noarch 6/6
已升级:dnf-plugins-core-4.0.21-16.el8.noarch python3-dnf-plugins-core-4.0.21-16.el8.noarch yum-utils-4.0.21-16.el8.noarch
完毕
[rootpig yum.repos.d]#然后使用yum-config-manager命令添加repo库
[rootpig yum.repos.d]# ls
CentOS-Stream-AppStream.repo CentOS-Stream-HighAvailability.repo CentOS-Stream-ResilientStorage.repo epel-testing-modular.repo
CentOS-Stream-BaseOS.repo CentOS-Stream-Media.repo CentOS-Stream-Sources.repo epel-testing.repo
CentOS-Stream-Debuginfo.repo CentOS-Stream-NFV.repo epel-modular.repo old
CentOS-Stream-Extras-common.repo CentOS-Stream-PowerTools.repo epel-playground.repo
CentOS-Stream-Extras.repo CentOS-Stream-RealTime.repo epel.repo
[rootpig yum.repos.d]# yum-config-manager --add-repo https://packages.clickhouse.com/rpm/clickhouse.repo
添加仓库自https://packages.clickhouse.com/rpm/clickhouse.repo
[rootpig yum.repos.d]# ls
CentOS-Stream-AppStream.repo CentOS-Stream-HighAvailability.repo CentOS-Stream-ResilientStorage.repo epel.repo
CentOS-Stream-BaseOS.repo CentOS-Stream-Media.repo CentOS-Stream-Sources.repo epel-testing-modular.repo
CentOS-Stream-Debuginfo.repo CentOS-Stream-NFV.repo clickhouse.repo epel-testing.repo
CentOS-Stream-Extras-common.repo CentOS-Stream-PowerTools.repo epel-modular.repo old
CentOS-Stream-Extras.repo CentOS-Stream-RealTime.repo epel-playground.repo
[rootpig yum.repos.d]#可以看到执行完毕后yum.repo.d目录下出现了clickhouse.repo文件其中记录了repo的baseurl和密钥url地址。
[rootpig yum.repos.d]# cat clickhouse.repo
[clickhouse-stable]
nameClickHouse - Stable Repository
baseurlhttps://packages.clickhouse.com/rpm/stable/
gpgkeyhttps://packages.clickhouse.com/rpm/stable/repodata/repomd.xml.key
gpgcheck0
repo_gpgcheck1
enabled1[clickhouse-lts]
nameClickHouse - LTS Repository
baseurlhttps://packages.clickhouse.com/rpm/lts/
gpgkeyhttps://packages.clickhouse.com/rpm/lts/repodata/repomd.xml.key
gpgcheck0
repo_gpgcheck1
enabled0
[rootpig yum.repos.d]#2. Debian/Ubuntu添加安装源 1添加镜像源准备 如CENTOS上的网络安全工具十六容器特色的Linux操作_lhyzws的博客-CSDN博客所提到的在执行官方给出的命令之前最好先执行一下apt update。否则可能导致apt-transport-https ca-certificates找不到安装包。 其中apt-transport-https ca-certificates我们已经比较熟悉是用来协助访问国内镜像源的组件dirmngr是用来管理和下载openpgp和x509证书的组件在前文中发挥这一作用的貌似是gnupg组件。本质上应该都是用来管理和使用镜像源的。
root5ebe7112320a:/# apt update
Ign:1 http://archive.ubuntu.com/ubuntu jammy InRelease
Ign:2 http://security.ubuntu.com/ubuntu jammy-security InRelease
Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]
………………
Get:18 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Fetched 24.9 MB in 7min 0s (59.4 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
2 packages can be upgraded. Run apt list --upgradable to see them.如果考虑更改镜像源则这几个支撑镜像源管理的包还是需要在更换之前安装的毕竟国内的一些镜像源依赖它们才能正确加载
root5ebe7112320a:/# apt install -y apt-transport-https ca-certificates dirmngr
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:gnupg gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm libassuan0 libksba8libldap-2.5-0 libldap-common libnpth0 libreadline8 libsasl2-2 libsasl2-modules libsasl2-modules-dblibsqlite3-0 openssl pinentry-curses readline-common
Suggested packages:dbus-user-session libpam-systemd pinentry-gnome3 tor parcimonie xloadimage scdaemonlibsasl2-modules-gssapi-mit | libsasl2-modules-gssapi-heimdal libsasl2-modules-ldap libsasl2-modules-otplibsasl2-modules-sql pinentry-doc readline-doc
The following NEW packages will be installed:apt-transport-https ca-certificates dirmngr gnupg gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-clientgpg-wks-server gpgconf gpgsm libassuan0 libksba8 libldap-2.5-0 libldap-common libnpth0 libreadline8libsasl2-2 libsasl2-modules libsasl2-modules-db libsqlite3-0 openssl pinentry-curses readline-common
0 upgraded, 25 newly installed, 0 to remove and 2 not upgraded.
Need to get 4817 kB of archives.
After this operation, 11.9 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 openssl amd64 3.0.2-0ubuntu1.7 [1183 kB]
……………………
Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 gpgconf amd64 2.2.27-3ubuntu2.1 [94.2 kB]
debconf: unable to initialize frontend: Readline
……………………
Updating certificates in /etc/ssl/certs...
124 added, 0 removed; done.
Setting up gpgconf (2.2.27-3ubuntu2.1) ...
Setting up gpg (2.2.27-3ubuntu2.1) ...
……………………
Setting up gpg-wks-client (2.2.27-3ubuntu2.1) ...
Setting up gnupg (2.2.27-3ubuntu2.1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.1) ...
Processing triggers for ca-certificates (20211016ubuntu0.22.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
root5ebe7112320a:/#镜像源更改请参考CENTOS上的网络安全工具十六容器特色的Linux操作_lhyzws的博客-CSDN博客不赘述。 2添加镜像源 首先是向/etc/apt/source.list.d目录下增加sourcelist文件。一般来说系统镜像源在/etc/apt/sources.list文件中描述应用镜像源放在source.list.d目录下
root5ebe7112320a:/etc/apt/sources.list.d# echo deb https://packages.clickhouse.com/deb stable main clickhouse.list
root5ebe7112320a:/etc/apt/sources.list.d# cat clickhouse.list
deb https://packages.clickhouse.com/deb stable main
root5ebe7112320a:/etc/apt/sources.list.d#其次是需要安装该镜像源的密钥
root5ebe7112320a:/etc/apt/sources.list.d# apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 8919F6BD2B48D754
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
Executing: /tmp/apt-key-gpghome.io9GX0It0w/gpg.1.sh --keyserver hkp://keyserver.ubuntu.com:80 --recv 8919F6BD2B48D754
gpg: key 8919F6BD2B48D754: public key ClickHouse Inc. Repositories Key packagesclickhouse.com imported
gpg: Total number processed: 1
gpg: imported: 1
root5ebe7112320a:/etc/apt/sources.list.d#二安装clickhouse服务器与客户端 1. Redhat/CentOS
[rootpig yum.repos.d]# yum install -y clickhouse-server clickhouse-client
ClickHouse - Stable Repository 18 B/s | 833 B 00:45
ClickHouse - Stable Repository 1.2 kB/s | 5.7 kB 00:04
导入 GPG 公钥 0x2B48D754:Userid: ClickHouse Inc. Repositories Key packagesclickhouse.com指纹: 3A9E A119 3A97 B548 BE14 57D4 8919 F6BD 2B48 D754来自: https://packages.clickhouse.com/rpm/stable/repodata/repomd.xml.key
ClickHouse - Stable Repository 4.2 kB/s | 86 kB 00:20
依赖关系解决。
软件包 架构 版本 仓库 大小安装:clickhouse-client x86_64 22.12.1.1752-1 clickhouse-stable 119 kclickhouse-server x86_64 22.12.1.1752-1 clickhouse-stable 145 k
安装依赖关系:clickhouse-common-static x86_64 22.12.1.1752-1 clickhouse-stable 252 M
事务概要安装 3 软件包
总下载253 M
安装大小721 M
下载软件包
(1/3): clickhouse-client-22.12.1.1752.x86_64.rpm 51 kB/s | 119 kB 00:02
(2/3): clickhouse-server-22.12.1.1752.x86_64.rpm 58 kB/s | 145 kB 00:02
[MIRROR] clickhouse-common-static-22.12.1.1752.x86_64.rpm: Curl error (28): Timeout was reached for https://packages.clickhouse.com/rpm/stable/clickhouse-common-static-22.12.1.1752.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds]
………………
chown -R clickhouse-bridge:clickhouse-bridge /usr/bin/clickhouse-odbc-bridgechown -R clickhouse-bridge:clickhouse-bridge /usr/bin/clickhouse-library-bridge
Password for default user is empty string. See /etc/clickhouse-server/users.xml and /etc/clickhouse-server/users.d to change it.
Setting capabilities for clickhouse binary. This is optional.chown -R clickhouse:clickhouse /etc/clickhouse-server
ClickHouse has been successfully installed.
Start clickhouse-server with:sudo clickhouse start
Start clickhouse-client with:clickhouse-client这里默认default user的口令是空字符串若需要设置口令需要后续更改配置文件。 2. Debian/Ubuntu 和yum不同更换镜像源后还是需要手动apt update一下。
root5ebe7112320a:/etc/apt/sources.list.d# apt update
Hit:1 https://mirrors.tuna.tsinghua.edu.cn/ubuntu jammy InRelease
Hit:2 https://mirrors.tuna.tsinghua.edu.cn/ubuntu jammy-updates InRelease
Hit:3 https://mirrors.tuna.tsinghua.edu.cn/ubuntu jammy-backports InRelease
Hit:4 https://mirrors.tuna.tsinghua.edu.cn/ubuntu jammy-security InRelease
Ign:5 https://packages.clickhouse.com/deb stable InRelease
Get:5 https://packages.clickhouse.com/deb stable InRelease [2484 B]
Get:6 https://packages.clickhouse.com/deb stable/main amd64 Packages [39.5 kB]
Fetched 42.0 kB in 17s (2405 B/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
2 packages can be upgraded. Run apt list --upgradable to see them.
W: https://packages.clickhouse.com/deb/dists/stable/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.然后再启动安装才不至于找不到安装包
root5ebe7112320a:/etc/apt/sources.list.d# apt install -y clickhouse-server clickhouse-client
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:clickhouse-common-static libcap2-bin libpam-cap
Suggested packages:clickhouse-common-static-dbg
The following NEW packages will be installed:clickhouse-client clickhouse-common-static clickhouse-server libcap2-bin libpam-cap
0 upgraded, 5 newly installed, 0 to remove and 2 not upgraded.
Need to get 265 MB of archives.
After this operation, 756 MB of additional disk space will be used.
Get:1 https://mirrors.tuna.tsinghua.edu.cn/ubuntu jammy/main amd64 libcap2-bin amd64 1:2.44-1build3 [26.0 kB]
Get:2 https://mirrors.tuna.tsinghua.edu.cn/ubuntu jammy/main amd64 libpam-cap amd64 1:2.44-1build3 [7932 B]
Get:3 https://packages.clickhouse.com/deb stable/main amd64 clickhouse-common-static amd64 22.12.1.1752 [265 MB]
9% [3 clickhouse-common-static 4215 kB/265 MB 2%] 282 kB/s 15min 25s
…………………… 与yum不同的是安装过程中会要求输入default用户的口令否则不会继续向下执行。如果不需要口令回车即可。 不论哪个版本安装结束后再/etc/init.d目录下都能够看到clickhouse-server的目录。
[rootpig init.d]# pwd
/etc/init.d
[rootpig init.d]# ls
clickhouse-server functions README
[rootpig init.d]#三测试 1. 启动服务 1systemctl启动 在可以使用systemctl的情况下直接使用systemctl进行管理维护
[rootpig init.d]# systemctl start clickhouse-server
[rootpig init.d]# systemctl status clickhouse-server.service
● clickhouse-server.service - ClickHouse Server (analytic DBMS for big data)Loaded: loaded (/usr/lib/systemd/system/clickhouse-server.service; enabled; vendor preset: disabled)Active: active (running) since Tue 2022-12-20 02:02:14 EST; 17s agoMain PID: 38925 (clickhouse-serv)Tasks: 219 (limit: 11047)Memory: 187.5MCGroup: /system.slice/clickhouse-server.service└─38925 /usr/bin/clickhouse-server --config/etc/clickhouse-server/config.xml --pid-file/run/clickhouse-server/clickhouse-serve
12月 20 02:02:10 pig systemd[1]: Starting ClickHouse Server (analytic DBMS for big data)...
12月 20 02:02:10 pig clickhouse-server[38925]: Processing configuration file /etc/clickhouse-server/config.xml.
12月 20 02:02:10 pig clickhouse-server[38925]: Logging trace to /var/log/clickhouse-server/clickhouse-server.log
12月 20 02:02:10 pig clickhouse-server[38925]: Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log
12月 20 02:02:13 pig clickhouse-server[38925]: Processing configuration file /etc/clickhouse-server/config.xml.
12月 20 02:02:13 pig clickhouse-server[38925]: Saved preprocessed configuration to /var/lib/clickhouse/preprocessed_configs/config.xml.
12月 20 02:02:13 pig clickhouse-server[38925]: Processing configuration file /etc/clickhouse-server/users.xml.
12月 20 02:02:13 pig clickhouse-server[38925]: Saved preprocessed configuration to /var/lib/clickhouse/preprocessed_configs/users.xml.
12月 20 02:02:14 pig systemd[1]: Started ClickHouse Server (analytic DBMS for big data).
[rootpig init.d]# systemctl enable clickhouse-server
Synchronizing state of clickhouse-server.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable clickhouse-server
[rootpig init.d]#2service命令启动 使用service命令实际和systemctl是一样的。在不能使用systemctl的场合大致也无法使用service。
[rootpig init.d]# service clickhouse-server start
[rootpig init.d]# service clickhouse-server status
/var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid 38925.
The process with pid 38925 is running.3命令行后台启动 在没有systemctl和service的情况下可以使用官方推荐的命令行后台启动方式
sudo /etc/init.d/clickhouse-server start 查看/etc/init.d/clickhouse-server的shell代码。实际上当参数为start时执行命令为 case $1 instart)service_or_func start enable_cron;;service_or_func函数的工作实际是如果systemctl可用直接调用systemctl start clickhouse-server若不可用调用start函数
service_or_func()
{if [ -x /bin/systemctl ] [ -f /etc/systemd/system/clickhouse-server.service ] [ -d /run/systemd/system ]; thensystemctl $1 $PROGRAMelse$1fi
}start函数
start()
{${CLICKHOUSE_GENERIC_PROGRAM} start --user ${CLICKHOUSE_USER} --pid-path ${CLICKHOUSE_PIDDIR} --config-path ${CLICKHOUSE_CONFDIR} --binary-path ${CLICKHOUSE_BINDIR}
}根据代码中变量定义该语句实际为
clickhouse start --user clickhouse --pid-path /var/run/clickhouse-server –config-path /etc /clickhouse-server –binary-path /usr/bin 之所以要拆到这一步是因为如果直接按照官方的指南需要sudo执行/etc/init.d/clickhouse-server start的话由于我的系统没有sudo直接执行会出现错误。
root5ebe7112320a:/etc/init.d# /etc/init.d /clickhouse-server startchown -R clickhouse: /var/run/clickhouse-server/
Will run sudo -u clickhouse /usr/bin/clickhouse-server --config-file /etc/clickhouse-server/config.xml --pid-file /var/run/clickhouse-server/clickhouse-server.pid --daemon
/bin/sh: 1: sudo: not found
Code: 302. DB::Exception: Child process was exited with return code 127. (CHILD_WAS_NOT_EXITED_NORMALLY) (version 22.12.1.1752 (official build))从脚本分析中容易发现clickhouse实际上要求程序以clickhouse这个用户启动。 比如看一看clickhouse相关的文件夹除了client以外 owner实际都是clickhouse这可能是其必须以clickhouse启动的原因吧。
root5ebe7112320a:/etc# ls -l
………………
drwxr-xr-x 2 root root 4096 Dec 20 04:46 clickhouse-client
drwx------ 4 clickhouse clickhouse 4096 Dec 20 04:11 clickhouse-server
………………直接执行clickhouse start –user clickhouse …… 一样不好使报错信息一样。虽然clickhouse是编译后的可执行文件但从报错信息看其内部似乎是仍然用sudo调用了同在bin下的可执行文件clickhouse-server注意不要和前面的sh文件弄混。 所以还是老实将sudo安装上吧
root72afdf9527a8:/usr/bin# apt install sudo
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:sudo
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 820 kB of archives.
After this operation, 2564 kB of additional disk space will be used.
Get:1 https://mirrors.tuna.tsinghua.edu.cn/ubuntu jammy-updates/main amd64 sudo amd64 1.9.9-1ubuntu2.1 [820 kB]
Fetched 820 kB in 8s (100 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package sudo.
(Reading database ... 5264 files and directories currently installed.)
Preparing to unpack .../sudo_1.9.9-1ubuntu2.1_amd64.deb ...
Unpacking sudo (1.9.9-1ubuntu2.1) ...
Setting up sudo (1.9.9-1ubuntu2.1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.1) ...
root72afdf9527a8:/usr/bin#果然安装以后不论是clickhouse直接执行还是通过clickhouse-server脚本执行都可以了
root72afdf9527a8:/usr/bin# clickhouse start --user clickhouse --pid-path /var/run/clickhouse-server --config-path /etc/clickhouse-server --binary-path /usr/binchown -R clickhouse: /var/run/clickhouse-server/
Will run sudo -u clickhouse /usr/bin/clickhouse-server --config-file /etc/clickhouse-server/config.xml --pid-file /var/run/clickhouse-server/clickhouse-server.pid --daemon
Waiting for server to start
Waiting for server to start
Server started
root72afdf9527a8:/usr/bin#4控制台启动 官方给出的控制台启动方法实际就是systemctl/service的服务启动方法的手动形式。
[rootpig system]# pwd
/usr/lib/systemd/system
[rootpig system]# cat clickhouse-server.service
[Unit]
DescriptionClickHouse Server (analytic DBMS for big data)
Requiresnetwork-online.target
# NOTE: that After/Wantstime-sync.target is not enough, you need to ensure
# that the time was adjusted already, if you use systemd-timesyncd you are
# safe, but if you use ntp or some other daemon, you should configure it
# additionaly.
Aftertime-sync.target network-online.target
Wantstime-sync.target[Service]
Typenotify# Switching off watchdog is very important for sd_notify to work correctly.
EnvironmentCLICKHOUSE_WATCHDOG_ENABLE0
Userclickhouse
Groupclickhouse
Restartalways
RestartSec30
RuntimeDirectoryclickhouse-server
ExecStart/usr/bin/clickhouse-server --config/etc/clickhouse-server/config.xml --pid-file/run/clickhouse-server/clickhouse-server.pid
# Minus means that this file is optional.
EnvironmentFile-/etc/default/clickhouse
LimitCOREinfinity
LimitNOFILE500000
CapabilityBoundingSetCAP_NET_ADMIN CAP_IPC_LOCK CAP_SYS_NICE CAP_NET_BIND_SERVICE[Install]
# ClickHouse should not start from the rescue shell (rescue.target).
WantedBymulti-user.target
[rootpig system]#使用控制台启动的命令则是
[rootpig system]# clickhouse-server --config-file/etc/clickhouse-server/config.xml 需要注意的是直接输入clickhouse-server执行的不是./clickhouse-server的shell脚本而是/usr/bin下的可执行文件。 2. 客户端连接 简单测试情况下只需要在本机启动clickhouse-client执行SELECT 1。如果执行情况如下表示安装成功
root72afdf9527a8:/etc/init.d# clickhouse-client
ClickHouse client version 22.12.1.1752 (official build).
Connecting to localhost:9000 as user default.
Connected to ClickHouse server version 22.12.1 revision 54461.
Warnings:* Linux transparent hugepages are set to always. Check /sys/kernel/mm/transparent_hugepage/enabled
72afdf9527a8 :) SELECT 1
SELECT 1
Query id: f946c4ec-7e21-425a-8574-67e2e86288af
┌─1─┐
│ 1 │
└───┘
1 row in set. Elapsed: 0.001 sec.
72afdf9527a8 :) quit三、VSCODE上的ClickHouse编程环境 个人感觉除了性能的优化意外clickhouse很成功的一点是和mysql太像了不仅是安装部署的流程还是使用的感觉相对于HBASE要友好很多。这种无门槛效应应当能够吸引到相当多的用户。 所以和mysql的VSCODE环境部署一样clickhouse的VSCODE环境部署别无二致甚至sqltools工具都能够通用唯一需要添加的只是一个clickhouse的驱动插件而已。 一本地部署 首先我们尝试在本地部署VSCODE的clickhouse使用环境。本地部署前提自然是clickhouse-server在本地启动clickhouse-client客户端要首先能用。
我们的本地环境是安装在Vmware中的一台CentOS-stream-8虚拟机如前所述过程安装好了clickhouse环境。并进一步如CENTOS上的网络安全工具五CODE来打个酱油_lhyzws的博客-CSDN博客所述安装了VSCODE。 下一步和CENTOS上的网络安全工具七MYSQL也不能少_lhyzws的博客-CSDN博客所述相似安装SQLTools插件 安装完成后左侧应该会多出一个数据库图标 在驱动安装部分和mysql不同的在于是要搜索安装clickhouse的驱动插件 安装完成后点击左侧SQLTools的图标栏 点击“”图标或者add new connection按钮 选择数据库驱动 clickhouse 如图配置连接参数。比如testCK为自己取的名字后期可以用在标识active connection上面localhost不能填IP地址IP地址代表远程连接不是本地连接了此时一定连不上。8123是默认端口号不动。数据库和用户名都可以用default一般来说这两者都存在。然后点击测试连接成功的话会有如图的绿色小框跳出否则为红色错误信息框。 若正确点击“save connection”即可。 插件会给出连接参数的json配置。然后就可以“CONNECT NOW”了。 创建一个sql查询并且Run on active connection即可得到如下结果 二远程部署 如果不希望在VMware的CentOS里部署VSCODE也可以使用windows上的VSCODE远程登录。 然而如在“本地部署”中所述。远程登录必须以IP地址来指明所使用的服务器而默认情况下clickhouse是只允许localhost登录的。 1. 打开远程登录权限 clickhouse的远程登陆权限在/etc/clickhouse-server/目录下的config.xml文件中配置
[rootpig clickhouse-server]# pwd
/etc/clickhouse-server
[rootpig clickhouse-server]# ls
config.d config.xml users.d users.xml
[rootpig clickhouse-server]#vim config.xml搜索listen字符串可以看到如下图描述 其中被注释的listen_host::/listen_host用于在IPv4/IPv6都支持的情况下放开远程登录权限listen_host0.0.0.0/listen用于放开仅支持IPv4情况下的远程登录权限。要放开的话取消注释即可。 由于config.xml是只读文件使用vim修改后需要使用:w!命令写入。 2. 配置VSCODE 1使用SQLTools插件 打开远程权限以后的操作就和本地登录没有两样了——安装SQL tools和clickhouse drvier插件配置连接参数——在这里就一定要填写clickhouse-server所在的主机IP地址了 2使用MySQL Database Manager 当然也可以选择其他的插件。例如我们在折腾MySQL的时候用过的MySQL Database Manager工具由weijan Chen提供。只不过这个工具有越来越多的功能开始收费了所以后来我就不怎么用了。 同样添加连接的时候会出现其所支持的所有引擎——果然收费的就是不一样clickhouse也在其中点选以后下面的连接选项就会配置成默认的样式。同样配置就好用户default密码为空。 四、Docker上的单节点部署 一Dockers上的Server部署 如第二章内容所述我们在Ubuntulatest22.04 jammy版本的镜像中按照上述的过程安装了clickhouse-server和clickhouse-client,然后将其导出成为clickhousepig镜像。 再次载入镜像时需要绑定监听端口。
C:\Users\lhyzwdocker run -it --name pig -p 9000:9000 -p 8123:8123 clickhouse:pig bash 下面我们按失败的方式先来一遍 按照虚拟机本地模式使用命令行方式启动当然需要事先安装sudo命令
root02aea9ee1a0e:/etc/clickhouse-server# sudo /etc/init.d/clickhouse-server start
/var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid 67.
Will run sudo -u clickhouse /usr/bin/clickhouse-server --config-file /etc/clickhouse-server/config.xml --pid-file /var/run/clickhouse-server/clickhouse-server.pid --daemon
Waiting for server to start
Server started结果是服务正常启动在容器内使用clickhouse-client也可以正常访问如果上文一样。但是没有查看到端口打开
rootc2bb84e9c057:/etc/clickhouse-server# netstat -ltpn
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
rootc2bb84e9c057:/etc/clickhouse-server# 自然从外部无法连接。 按照虚拟机远程模式更改/etc/clickhouse-server/config.xml
sed -i s!-- listen_host::/listen_host --listen_host::/listen_hostg config.xml
sed -i s!-- listen_host0.0.0.0/listen_host --listen_host0.0.0.0/listen_hostg config.xml然后再sudo启动结果是clickhouse-client无法以localhost:9000方式进行本地连接监听端口仍然没启动外面也无法远程连接。 正确方式 具体原因一头雾水……不过幸好有clickhouse官方镜像可以参考。经过一番比对发现只需要改动一个地方即/etc/clickhouse-server/config.d文件夹下构造一个名为docker_related_config.xml的文件即可内容如下所示
rootf5f12d60e478:/etc/clickhouse-server/config.d# cat docker_related_config.xml
clickhouse!-- Listen wildcard address to allow accepting connections from other containers and host network. --listen_host::/listen_hostlisten_host0.0.0.0/listen_hostlisten_try1/listen_try!--loggerconsole1/console/logger--
/clickhouse
rootf5f12d60e478:/etc/clickhouse-server/config.d#改动的内容和在/etc/clickhouse-server.xml文件中的改动基本一致除了listen_try这个选项。官方的注释是这么解释的当值为0时代表“Dont exit if IPv6 or IPv4 networks are unavailable while trying to listen”。这个改完后重新sudo启动顺滑得一匹
C:\Users\lhyzwdocker run -it -p 8123:8123 -p9000:9000 --name pig1 clickhouse:pig bash
rootf5f12d60e478:/# sudo /etc/init.d/clickhouse-server start
/var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid 67.
Will run sudo -u clickhouse /usr/bin/clickhouse-server --config-file /etc/clickhouse-server/config.xml --pid-file /var/run/clickhouse-server/clickhouse-server.pid --daemon
Waiting for server to start
Server started
rootf5f12d60e478:/# netstat -ltpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:8123 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:9009 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:9000 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:9004 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:9005 0.0.0.0:* LISTEN -原因不知。也许因为docker的网络驱动问题。猜测有可能clickhouse是绕开了传统网络驱动栈所以需要对docker做一些专门的处理……。总之在config.d下添加这个文件把listen_try设置为1倒是发挥了作用。 如此配置后在VSCODE中使用localhost、127.0.0.1和WSL子系统的IP地址都可以连接成功相当给力。 二直接使用官方镜像 1.加载镜像 直接使用官方镜像的情况比较简单参考clickhouse/clickhouse-server - Docker Image | Docker Hub即可。以官方镜像clickhouse/clickhouse-server为例 启动clickhouse-server,可根据官方对镜像使用方法的介绍不过端口我原样映射了太多了记不住。另外--ulimit nofile262144:262144是用于Linux内核优化的设置控制打开的文件数量。Linux内核默认对每个用户设置最大打开文件数为1024在clickhouse的场景内显然是不够的。当然只测试连接的情况不设置这个参数也没有问题。
C:\Users\lhyzwdocker run -d -p 8123:8123 -p 9000:9000 --name pig --ulimit nofile262144:262144 clickhouse/clickhouse-server
c617624c8629919a135f739a35e74c1c376c4214db74f0099a5837072e6ce6f0
C:\Users\lhyzwdocker exec -it pig bash
rootc617624c8629:/# clickhouse-client
ClickHouse client version 22.12.1.1752 (official build).
c617624c8629 :) quit
Bye.启动后可以直接挂进容器先本地测试一下是否能够连接……一般是绝对可以的。 然后如上面我们自己弄的容器一样VSCODE连接之一般也是没有问题的。 2. 指定数据存储位置 clickhouse有两个位置是用户比较关心的。一个是/var/lib/clickhouse是clickhouse用来存放数据的地方另一个是/var/log/clickhouse-server是用来存放日志的地方。如果需要明确指定这两个目录可以用-v 参数进行映射。 另外还有3个涉及到容器配置的文件或目录可能需要统一映射分别是 /etc/clickhouse-server/config.d/*.xml配置相关这个我们是cp进去的 /etc/clickhouse-server/user.d/*.xml用户相关用户口令可以在这里添加 /docker-entrypoint-initdb.d/ 用于存放数据库初始化脚本的地方。
