zkcopy
Tool for fast copying ZooKeeper data between different clusters. Originally it was developed for copying big volumes of configuration over WAN.
GitHub
https://github.com/ksprojects/zkcopy
Build
Requires apache maven 3.
mvn clean install
Usage
java -jar target/zkcopy.jar --source server:port/path --target server:port/path
With docker, use following commands:
docker pull ksprojects/zkcopy
docker run --rm -it ksprojects/zkcopy --source server:port/path --target server:port/path
Options
迁移准备工作
下载安装
devops@devopsdeMacBook-Pro ~ % git clone https://github.com/ksprojects/zkcopy.git Cloning into 'zkcopy'... remote: Enumerating objects: 409, done. remote: Total 409 (delta 0), reused 0 (delta 0), pack-reused 409 Receiving objects: 100% (409/409), 1.40 MiB | 86.00 KiB/s, done. Resolving deltas: 100% (143/143), done. devops@devopsdeMacBook-Pro ~ % cd zkcopy devops@devopsdeMacBook-Pro zkcopy % mvn clean install
查看源zk数据
[centos@us-prod-sre-zkserver-1 ~]$ cd /data/app/apache-zookeeper-3.5.7-bin/bin [centos@us-prod-sre-zkserver-1 bin]$ ./zkCli.sh -server 127.0.0.1:2181 Connecting to 127.0.0.1:2181 2022-07-26 08:18:50,216 [myid:] - INFO [main:Environment@109] - Client environment:zookeeper.version=3.5.7-f0fdd52973d373ffd9c86b81d99842dc2c7f660e, built on 02/10/2020 11:30 GMT 2022-07-26 08:18:50,220 [myid:] - INFO [main:Environment@109] - Client environment:host.name=zk-server 2022-07-26 08:18:50,220 [myid:] - INFO [main:Environment@109] - Client environment:java.version=13.0.1 2022-07-26 08:18:50,224 [myid:] - INFO [main:Environment@109] - Client environment:java.vendor=Oracle Corporation 2022-07-26 08:18:50,224 [myid:] - INFO [main:Environment@109] - Client environment:java.home=/data/app/jdk-13.0.1 2022-07-26 08:18:50,224 [myid:] - INFO [main:Environment@109] - Client environment:java.class.path=/data/app/apache-zookeeper-3.5.7-bin/bin/../zookeeper-server/target/classes:/data/app/apache-zookeeper-3.5.7-bin/bin/../build/classes:/data/app/apache-zookeeper-3.5.7-bin/bin/../zookeeper-server/target/lib/*.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../build/lib/*.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/zookeeper-jute-3.5.7.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/zookeeper-3.5.7.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/slf4j-log4j12-1.7.25.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/slf4j-api-1.7.25.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-transport-native-unix-common-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-transport-native-epoll-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-transport-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-resolver-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-handler-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-common-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-codec-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/netty-buffer-4.1.45.Final.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/log4j-1.2.17.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/json-simple-1.1.1.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jline-2.11.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-util-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-servlet-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-server-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-security-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-io-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jetty-http-9.4.24.v20191120.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/javax.servlet-api-3.1.0.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jackson-databind-2.9.10.2.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jackson-core-2.9.10.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/jackson-annotations-2.9.10.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/commons-cli-1.2.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../lib/audience-annotations-0.5.0.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../zookeeper-*.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../zookeeper-server/src/main/resources/lib/*.jar:/data/app/apache-zookeeper-3.5.7-bin/bin/../conf:.:/data/app/jdk-13.0.1//lib:/data/app/jdk-13.0.1//jre/lib 2022-07-26 08:18:50,224 [myid:] - INFO [main:Environment@109] - Client environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib 2022-07-26 08:18:50,224 [myid:] - INFO [main:Environment@109] - Client environment:java.io.tmpdir=/tmp 2022-07-26 08:18:50,225 [myid:] - INFO [main:Environment@109] - Client environment:java.compiler=<NA> 2022-07-26 08:18:50,225 [myid:] - INFO [main:Environment@109] - Client environment:os.name=Linux 2022-07-26 08:18:50,225 [myid:] - INFO [main:Environment@109] - Client environment:os.arch=amd64 2022-07-26 08:18:50,225 [myid:] - INFO [main:Environment@109] - Client environment:os.version=3.10.0-1062.12.1.el7.x86_64 2022-07-26 08:18:50,225 [myid:] - INFO [main:Environment@109] - Client environment:user.name=centos 2022-07-26 08:18:50,226 [myid:] - INFO [main:Environment@109] - Client environment:user.home=/home/centos 2022-07-26 08:18:50,226 [myid:] - INFO [main:Environment@109] - Client environment:user.dir=/data/app/apache-zookeeper-3.5.7-bin/bin 2022-07-26 08:18:50,226 [myid:] - INFO [main:Environment@109] - Client environment:os.memory.free=241MB 2022-07-26 08:18:50,229 [myid:] - INFO [main:Environment@109] - Client environment:os.memory.max=256MB 2022-07-26 08:18:50,229 [myid:] - INFO [main:Environment@109] - Client environment:os.memory.total=248MB 2022-07-26 08:18:50,233 [myid:] - INFO [main:ZooKeeper@868] - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@2d6e8792 2022-07-26 08:18:50,242 [myid:] - INFO [main:X509Util@79] - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation 2022-07-26 08:18:50,255 [myid:] - INFO [main:ClientCnxnSocket@237] - jute.maxbuffer value is 4194304 Bytes 2022-07-26 08:18:50,263 [myid:] - INFO [main:ClientCnxn@1653] - zookeeper.request.timeout value is 0. feature enabled= Welcome to ZooKeeper! 2022-07-26 08:18:50,297 [myid:127.0.0.1:2181] - INFO [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1112] - Opening socket connection to server mw-server-1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2022-07-26 08:18:50,339 [myid:127.0.0.1:2181] - INFO [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@959] - Socket connection established, initiating session, client: /127.0.0.1:44536, server: mw-server-1/127.0.0.1:2181 2022-07-26 08:18:50,359 [myid:127.0.0.1:2181] - INFO [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1394] - Session establishment complete on server mw-server-1/127.0.0.1:2181, sessionid = 0x109291b7f8510f9, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: 127.0.0.1:2181(CONNECTED) 0] ls / [dubbo, lock, signal_api_test, test, zookeeper]
同步zk数据
nohup java -jar zkcopy.jar --source 10.0.3.93:2181/dubbo --target 10.0.3.113:2181,10.0.3.121:2181,10.0.3.217:2181/dubbo & sleep 1 nohup java -jar zkcopy.jar --source 10.0.3.93:2181/lock --target 10.0.3.113:2181,10.0.3.121:2181,10.0.3.217:2181/lock & sleep 1 nohup java -jar zkcopy.jar --source 10.0.3.93:2181/signal_api_test --target 10.0.3.113:2181,10.0.3.121:2181,10.0.3.217:2181/signal_api_test & sleep 1 nohup java -jar zkcopy.jar --source 10.0.3.93:2181/test --target 10.0.3.113:2181,10.0.3.121:2181,10.0.3.217:2181/test &
验证数据
[centos@us-prod-sre-zookeeper-3 bin]$ ./zkCli.sh -server 127.0.0.1:2181 zk: 127.0.0.1:2181(CONNECTED) 6] ls / [dubbo, lock, signal_api_test, test, zookeeper] [zk: 127.0.0.1:2181(CONNECTED) 7] get /dubbo 10.0.2.137 [zk: 127.0.0.1:2181(CONNECTED) 8] get /signal_api_test mydata [zk: 127.0.0.1:2181(CONNECTED) 9] quit WATCHER::
继续阅读
评论