zookeeper数据迁移zkcopy

DevOps Zookeeper评论2,676字数 7631阅读25分26秒阅读模式

zkcopy

Tool for fast copying ZooKeeper data between different clusters. Originally it was developed for copying big volumes of configuration over WAN.

GitHub

https://github.com/ksprojects/zkcopy

Build

Requires apache maven 3.

mvn clean install

Usage

java -jar target/zkcopy.jar --source server:port/path --target server:port/path

With docker, use following commands:

docker pull ksprojects/zkcopy
docker run --rm -it ksprojects/zkcopy --source server:port/path --target server:port/path

Options

zookeeper数据迁移zkcopy

迁移准备工作

下载安装
devops@devopsdeMacBook-Pro ~ % git clone https://github.com/ksprojects/zkcopy.git
Cloning into 'zkcopy'...
remote: Enumerating objects: 409, done.
remote: Total 409 (delta 0), reused 0 (delta 0), pack-reused 409
Receiving objects: 100% (409/409), 1.40 MiB | 86.00 KiB/s, done.
Resolving deltas: 100% (143/143), done.
devops@devopsdeMacBook-Pro ~ % cd zkcopy 
devops@devopsdeMacBook-Pro zkcopy % mvn clean install
查看源zk数据
[centos@us-prod-sre-zkserver-1 ~]$ cd /data/app/apache-zookeeper-3.5.7-bin/bin
[centos@us-prod-sre-zkserver-1 bin]$ ./zkCli.sh -server 127.0.0.1:2181
Connecting to 127.0.0.1:2181
2022-07-26 08:18:50,216 [myid:] - INFO  [main:Environment@109] - Client environment:zookeeper.version=3.5.7-f0fdd52973d373ffd9c86b81d99842dc2c7f660e, built on 02/10/2020 11:30 GMT
2022-07-26 08:18:50,220 [myid:] - INFO  [main:Environment@109] - Client environment:host.name=zk-server
2022-07-26 08:18:50,220 [myid:] - INFO  [main:Environment@109] - Client environment:java.version=13.0.1
2022-07-26 08:18:50,224 [myid:] - INFO  [main:Environment@109] - Client environment:java.vendor=Oracle Corporation
2022-07-26 08:18:50,224 [myid:] - INFO  [main:Environment@109] - Client environment:java.home=/data/app/jdk-13.0.1
2022-07-26 08:18:50,224 [myid:] - INFO  [main:Environment@109] - Client environment:java.class.path=/data/app/apache-zookeeper-3.5.7-bin/bin/../zookeeper-server/target/classes:/data/app/apache-zookeeper-3.5.7-bin/bin/../build/classes:/data/app/apache-zookeeper-3.5.7-bin/bin/../zookeeper-server/target/lib/*.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../build/lib/*.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/zookeeper-jute-3.5.7.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/zookeeper-3.5.7.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/slf4j-log4j12-1.7.25.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/slf4j-api-1.7.25.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-transport-native-unix-common-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-transport-native-epoll-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-transport-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-resolver-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-handler-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-common-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-codec-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-buffer-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/log4j-1.2.17.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/json-simple-1.1.1.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jline-2.11.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-util-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-servlet-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-server-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-security-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-io-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-http-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/javax.servlet-api-3.1.0.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jackson-databind-2.9.10.2.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jackson-core-2.9.10.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jackson-annotations-2.9.10.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/commons-cli-1.2.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/audience-annotations-0.5.0.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../zookeeper-*.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../zookeeper-server/src/main/resources/lib/*.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../conf:.:/data/app/jdk-13.0.1//lib:/data/app/jdk-13.0.1//jre/lib
2022-07-26 08:18:50,224 [myid:] - INFO  [main:Environment@109] - Client environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
2022-07-26 08:18:50,224 [myid:] - INFO  [main:Environment@109] - Client environment:java.io.tmpdir=/tmp
2022-07-26 08:18:50,225 [myid:] - INFO  [main:Environment@109] - Client environment:java.compiler=<NA>
2022-07-26 08:18:50,225 [myid:] - INFO  [main:Environment@109] - Client environment:os.name=Linux
2022-07-26 08:18:50,225 [myid:] - INFO  [main:Environment@109] - Client environment:os.arch=amd64
2022-07-26 08:18:50,225 [myid:] - INFO  [main:Environment@109] - Client environment:os.version=3.10.0-1062.12.1.el7.x86_64
2022-07-26 08:18:50,225 [myid:] - INFO  [main:Environment@109] - Client environment:user.name=centos
2022-07-26 08:18:50,226 [myid:] - INFO  [main:Environment@109] - Client environment:user.home=/home/centos
2022-07-26 08:18:50,226 [myid:] - INFO  [main:Environment@109] - Client environment:user.dir=/data/app/apache-zookeeper-3.5.7-bin/bin
2022-07-26 08:18:50,226 [myid:] - INFO  [main:Environment@109] - Client environment:os.memory.free=241MB
2022-07-26 08:18:50,229 [myid:] - INFO  [main:Environment@109] - Client environment:os.memory.max=256MB
2022-07-26 08:18:50,229 [myid:] - INFO  [main:Environment@109] - Client environment:os.memory.total=248MB
2022-07-26 08:18:50,233 [myid:] - INFO  [main:ZooKeeper@868] - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@2d6e8792
2022-07-26 08:18:50,242 [myid:] - INFO  [main:X509Util@79] - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2022-07-26 08:18:50,255 [myid:] - INFO  [main:ClientCnxnSocket@237] - jute.maxbuffer value is 4194304 Bytes
2022-07-26 08:18:50,263 [myid:] - INFO  [main:ClientCnxn@1653] - zookeeper.request.timeout value is 0. feature enabled=
Welcome to ZooKeeper!
2022-07-26 08:18:50,297 [myid:127.0.0.1:2181] - INFO  [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1112] - Opening socket connection to server mw-server-1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2022-07-26 08:18:50,339 [myid:127.0.0.1:2181] - INFO  [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@959] - Socket connection established, initiating session, client: /127.0.0.1:44536, server: mw-server-1/127.0.0.1:2181
2022-07-26 08:18:50,359 [myid:127.0.0.1:2181] - INFO  [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1394] - Session establishment complete on server mw-server-1/127.0.0.1:2181, sessionid = 0x109291b7f8510f9, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: 127.0.0.1:2181(CONNECTED) 0] ls /
[dubbo, lock, signal_api_test, test, zookeeper]
同步zk数据
nohup java -jar zkcopy.jar --source 10.0.3.93:2181/dubbo --target 10.0.3.113:2181,10.0.3.121:2181,10.0.3.217:2181/dubbo &
sleep 1
nohup java -jar zkcopy.jar --source 10.0.3.93:2181/lock --target 10.0.3.113:2181,10.0.3.121:2181,10.0.3.217:2181/lock &
sleep 1
nohup java -jar zkcopy.jar --source 10.0.3.93:2181/signal_api_test --target 10.0.3.113:2181,10.0.3.121:2181,10.0.3.217:2181/signal_api_test &
sleep 1
nohup java -jar zkcopy.jar --source 10.0.3.93:2181/test --target 10.0.3.113:2181,10.0.3.121:2181,10.0.3.217:2181/test &
验证数据
[centos@us-prod-sre-zookeeper-3 bin]$ ./zkCli.sh -server 127.0.0.1:2181
zk: 127.0.0.1:2181(CONNECTED) 6] ls /
[dubbo, lock, signal_api_test, test, zookeeper]
[zk: 127.0.0.1:2181(CONNECTED) 7] get /dubbo
10.0.2.137
[zk: 127.0.0.1:2181(CONNECTED) 8] get /signal_api_test
mydata
[zk: 127.0.0.1:2181(CONNECTED) 9] quit

WATCHER::

 

继续阅读
DevOps
  • 本文由 发表于 2022年7月26日 15:15:26
  • 除非特殊声明,本站文章均为原创,转载请务必保留本文链接
Zookeeper3.5.7集群部署 Zookeeper

Zookeeper3.5.7集群部署

简述 ZooKeeper 是 Apache 的一个顶级项目,为分布式应用提供高效、高可用的分布式协调服务。 ZooKeeper本质上是一个分布式的小文件存储系统。提供类似于文件系统目录树方式的...
Zookeeper3.8集群 Zookeeper

Zookeeper3.8集群

安装jdk 安装文档 集群规划 zookeper集群至少需要3台服务器,并且强烈建议使用奇数个服务器。因为zookeeper通过半数机制来判断整个集群是否可用,也就是说集群中有一半以...
评论  0  访客  0

发表评论