在上一篇博客中我們聊到了redis的高可用組件sentinel的相關(guān)配置，回顧請參考
https://www.cnblogs.com/qiuhom-1874/p/13429776.html；sentinel在redis主從同步架構(gòu)中主要起到了監(jiān)控集群master是否正常，如果master不正常，或者宕機，那么sentinel會提升一個slave當選新的master，從而保證了redis服務(wù)的正常使用；但對于redis的單機寫入問題還是一直存在；在sentinel+主從同步架構(gòu)中，程序?qū)憯?shù)據(jù)，始終是把讀寫請求發(fā)送給master（當然，如果有語句路由器讀寫請求是可以分開的）；這樣一來對于master來講它就承擔(dān)了所有的寫操作，很顯然這種在寫操作非常的頻繁的場景，單臺master肯定無法承受這么大的壓力；為了解決單機master寫入數(shù)據(jù)的瓶頸問題， redis 官方在 redis 3.0 版本之后推出了無中心架構(gòu)的 redis cluster 機制；所謂無中心架構(gòu)的cluster意思是集群中的每一個成員都是中心節(jié)點，每個成員擁有整個集群的狀態(tài)元數(shù)據(jù)，但各自又只存儲整個集群的部分數(shù)據(jù)；對于redis cluster這種架構(gòu)來講，所有redis節(jié)點使用（ping機制）互聯(lián)，如果集群中某個節(jié)點的失敗，是需要整個集群中超過半數(shù)的節(jié)點監(jiān)測都失效才算真正的失效；客戶端不需要 proxy 即可直接連接 redis，應(yīng)用程序需要寫全部的 redis 服務(wù)器 IP。redis cluster 把所有的 redis node 映射到 0-16383 個槽位(slot)上，讀寫需要到指定的 redis node 上進行操作，因此有多少個 reids node 相當于 redis 并發(fā)擴展了多少倍。Redis cluster 預(yù)先分配 16384 個(slot)槽位，當需要在 redis 集群中寫入一個 key -value 的時候，會使用 CRC16(key) mod 16384 之后的值，決定將 key 寫入值哪一個槽位從而決定寫入哪一個 Redis 節(jié)點上，從而有效解決redis單機讀寫瓶頸。如下所示
提示：客戶端每次讀寫都要經(jīng)過CRC16對讀寫的KEY做hash計算，然后把hash后的結(jié)果再和16384做取模運算，最終結(jié)果會落到0-16383的某一個槽位上；我們事先在定義集群時，就把0-5460號槽位都分給了master01，意思就是只要讀寫的數(shù)據(jù)的KEY通過CRC16hash計算取得的結(jié)果%16384，最后的結(jié)果在0-5460這個范圍內(nèi)，就把本次的讀/寫請求調(diào)度到master01上；以此類推，范圍在5461-10922就把讀/寫請求調(diào)度到master02,范圍在10923-16383就把讀/寫請求調(diào)度到master03;通過上面的圖我們也能看出各個master都只保存整個集群的一部分數(shù)據(jù)，如果某一個master宕機，那么就會導(dǎo)致對應(yīng)master上分的槽位對應(yīng)數(shù)據(jù)丟失；為了解決各個master單點的問題，我們還需要對每個master做主從，如下圖
提示：為了防止某個master宕機后導(dǎo)致數(shù)據(jù)丟失和寫操作不能執(zhí)行，對應(yīng)slave應(yīng)該能夠及時的提升為master，類似sentinel的功能，但是我們不需要在部署sentinel了，因為redis cluster就支持主從切換；這樣一來我們的集群就變成了3主3從的集群；接下我們就準備一個實驗環(huán)境來部署上面的架構(gòu)；
部署redis集群
環(huán)境說明
角色ip地址端口
Master01192.168.0.416379
Slave01192.168.0.426380
Master02192.168.0.426379
Slave02192.168.0.436380
Master03192.168.0.436379
Slave03192.168.0.416380
準備啟動3臺服務(wù)器，每個服務(wù)器上啟動兩個redis實例，端口分別是6379（master）和6380（slave)，為了防止master和slave落在同一個節(jié)點，有意將他們交叉組合，如上表格；
創(chuàng)建redis cluster我們需要注意每個 redis node 節(jié)點盡量采用相同的硬件配置，相同的密碼；所有 redis 服務(wù)器必須沒有任何數(shù)據(jù)；
在node01上編譯安裝redis，有關(guān)redis的編譯安裝和部署請參考
https://www.cnblogs.com/qiuhom-1874/p/13378138.html；
創(chuàng)建目錄結(jié)構(gòu)
[root@node01 ~]# tree /usr/local/redis//usr/local/redis/├── 6379│ ├── etc│ │ ├── redis.conf│ │ └── sentinel.conf│ ├── logs│ └── run├── 6380│ ├── etc│ │ ├── redis.conf│ │ └── sentinel.conf│ ├── logs│ └── run└── bin ├── redis-benchmark ├── redis-check-aof ├── redis-check-rdb ├── redis-cli ├── redis-sentinel -> redis-server └── redis-server9 directories, 10 files[root@node01 ~]#
提示：在/usr/local/redis目錄下分別創(chuàng)建6379和6380,然后在其下都創(chuàng)建etc,logs,run 等子目錄，如上所示
修改配置文件
[root@node01 ~]# grep -E "^(bind|requirepass|masterauth|cluster)" /usr/local/redis/6379/etc/redis.conf bind 0.0.0.0masterauth adminrequirepass admincluster-enabled yescluster-config-file redis-cluster_6379.conf[root@node01 ~]# grep -E "^(bind|requirepass|masterauth|cluster)" /usr/local/redis/6380/etc/redis.conf bind 0.0.0.0masterauth adminrequirepass admincluster-enabled yescluster-config-file redis-cluster_6380.conf[root@node01 ~]#
提示：6379和6380的配置文件內(nèi)容都是一樣的，唯一區(qū)別是端口不同，主要修改bind 監(jiān)聽地址，設(shè)置密碼，開啟集群以及指定對應(yīng)集群生成的配置文件名稱；
把node01上的/usr/local/redis復(fù)制給node02和node03并保存到/usr/local/目錄下
提示：我上面的三臺主機都是做了免密登錄，所以互相拷貝數(shù)據(jù)不需要輸入密碼；有關(guān)免密登錄的說明和配置可以參考
https://www.cnblogs.com/qiuhom-1874/p/11783371.html；
分別在node01 node02 node03上啟動redis
到此我們就把6個redis實例啟動了；接下來我們來創(chuàng)建集群；Redis 3 和 4 版本需要使用到集群管理工具 redis-trib.rb，這個工具是 redis 官方推出的管理 redis 集群的工具，集成在redis 的源碼 src 目錄下，是基于 redis 提供的集群命令封裝成簡單、便捷、實用的操作工具，redis-trib.rb是 redis 作者用 ruby 開發(fā)完成的，所以我們要使用redis-trib.rb首先要解決ruby環(huán)境的問題；
安裝ruby環(huán)境
[root@node01 ~]# rzrz waiting to receive. zmodem trl+C ? 100% 15621 KB 15621 KB/s 00:00:01 0 Errors[root@node01 ~]# lsredis-4.0.9.tar.gz ruby-2.5.5.tar.gz[root@node01 ~]# tar xf ruby-2.5.5.tar.gz -C /usr/local/src/[root@node01 ~]# cd /usr/local/src/[root@node01 src]# cd ruby-2.5.5/[root@node01 ruby-2.5.5]# ./configure checking for ruby... falsechecking build system type... x86_64-pc-linux-gnuchecking host system type... x86_64-pc-linux-gnuchecking target system type... x86_64-pc-linux-gnuchecking for gcc... gccchecking whether the C compiler works... yeschecking for C compiler default output file name... a.outchecking for suffix of executables... checking whether we are cross compiling... nochecking for suffix of object files... ochecking whether we are using the GNU C compiler... yeschecking whether gcc accepts -g... yeschecking for gcc option to accept ISO C89... none neededchecking for g++... g++checking whether we are using the GNU C++ compiler... yes……省略部分內(nèi)容……checking for setjmp type... __builtin_setjmpchecking for prefix of external symbols... NONEchecking pthread.h usability... yeschecking pthread.h presence... yeschecking for pthread.h... yeschecking if make is GNU make... yes.ext/include/x86_64-linux/ruby/config.h updatedconfigure: ruby library version = 2.5.0configure: creating ./config.statusconfig.status: creating GNUmakefileconfig.status: creating Makefileconfig.status: creating ruby-2.5.pc---Configuration summary for ruby version 2.5.5 * Installation prefix: /usr/local * exec prefix: ${prefix} * arch: x86_64-linux * site arch: ${arch} * RUBY_BASE_NAME: ruby * ruby lib prefix: ${libdir}/${RUBY_BASE_NAME} * site libraries path: ${rubylibprefix}/${sitearch} * vendor path: ${rubylibprefix}/vendor_ruby * target OS: linux * compiler: gcc * with pthread: yes * enable shared libs: no * dynamic library ext: so * CFLAGS: ${optflags} ${debugflags} ${warnflags} * LDFLAGS: -L. -fstack-protector -rdynamic -Wl,-export-dynamic * optflags: -O3 * debugflags: -ggdb3 * warnflags: -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wno-tautological-compare -Wno-parentheses-equality -Wno-constant-logical-operand -Wno-self-assign -Wunused-variable -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wdeclaration-after-statement -Wimplicit-function-declaration -Wdeprecated-declarations -Wno-packed-bitfield-compat -Wsuggest-attribute=noreturn -Wsuggest-attribute=format * strip command: strip -S -x * install doc: yes * man page type: doc---[root@node01 ruby-2.5.5]#
提示：redis-trib.rb這個工具在集群任意一臺安裝即可，不需要每臺都安裝；
編譯
[root@node01 ruby-2.5.5]# make -j 2 CC = gcc LD = ld LDSHARED = gcc -shared CFLAGS = -O3 -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wno-tautological-compare -Wno-parentheses-equality -Wno-constant-logical-operand -Wno-self-assign -Wunused-variable -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wdeclaration-after-statement -Wimplicit-function-declaration -Wdeprecated-declarations -Wno-packed-bitfield-compat -Wsuggest-attribute=noreturn -Wsuggest-attribute=format -std=gnu99 XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE CPPFLAGS = -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/10.0.0 DLDFLAGS = -fstack-protector -pie SOLIBS = LANG = en_US.UTF-8 LC_ALL = LC_CTYPE = gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)Copyright (C) 2015 Free Software Foundation, Inc.This is free software; see the source for copying conditions. There is NOwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.compiling ./main.ccompiling dmydln.ccompiling miniinit.ccompiling dmyext.ccompiling miniprelude.cmaking dummy probes.hcompiling bignum.ccompiling class.ccompiling compar.ccompiling compile.ccompiling complex.ccompiling cont.ccompiling debug.ccompiling debug_counter.ccompiling dir.ccompiling dln_find.ccompiling encoding.ccompiling enum.ccompiling enumerator.c……省略部分內(nèi)容……*** Fix the problems, then remove these directories and try again if you want.make[1]: Leaving directory `/usr/local/src/ruby-2.5.5'Generating RDoc documentationParsing sources...100% [871/871] vsnprintf.c Generating RI format into /usr/local/src/ruby-2.5.5/.ext/rdoc... Files: 871 Classes: 1324 ( 565 undocumented) Modules: 286 ( 121 undocumented) Constants: 2181 ( 555 undocumented) Attributes: 1066 ( 251 undocumented) Methods: 10080 (2161 undocumented) Total: 14937 (3653 undocumented) 75.54% documented Elapsed: 25.0s[root@node01 ruby-2.5.5]#
安裝ruby環(huán)境
[root@node01 ruby-2.5.5]# make install CC = gcc LD = ld LDSHARED = gcc -shared CFLAGS = -O3 -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wno-tautological-compare -Wno-parentheses-equality -Wno-constant-logical-operand -Wno-self-assign -Wunused-variable -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wdeclaration-after-statement -Wimplicit-function-declaration -Wdeprecated-declarations -Wno-packed-bitfield-compat -Wsuggest-attribute=noreturn -Wsuggest-attribute=format -std=gnu99 XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE CPPFLAGS = -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/10.0.0 DLDFLAGS = -fstack-protector -pie SOLIBS = LANG = en_US.UTF-8 LC_ALL = LC_CTYPE = gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)Copyright (C) 2015 Free Software Foundation, Inc.This is free software; see the source for copying conditions. There is NOwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.generating enc.mkmaking srcs under encmake[1]: Entering directory `/usr/local/src/ruby-2.5.5'make[1]: Nothing to be done for `srcs'.make[1]: Leaving directory `/usr/local/src/ruby-2.5.5'generating transdb.htransdb.h unchangedgenerating makefiles ext/configure-ext.mkext/configure-ext.mk updatedmake[1]: Entering directory `/usr/local/src/ruby-2.5.5'make[1]: Leaving directory `/usr/local/src/ruby-2.5.5'generating makefile exts.mkexts.mk updatedmake[1]: Entering directory `/usr/local/src/ruby-2.5.5'make[2]: Entering directory `/usr/local/src/ruby-2.5.5/ext/fcntl'make[2]: Leaving directory `/usr/local/src/ruby-2.5.5/ext/fcntl'make[2]: Entering directory `/usr/local/src/ruby-2.5.5/ext/rubyvm'……省略部分內(nèi)容……installing binary commands: /usr/local/bininstalling base libraries: /usr/local/libinstalling arch files: /usr/local/lib/ruby/2.5.0/x86_64-linuxinstalling pkgconfig data: /usr/local/lib/pkgconfiginstalling command scripts: /usr/local/bininstalling library scripts: /usr/local/lib/ruby/2.5.0installing common headers: /usr/local/include/ruby-2.5.0installing manpages: /usr/local/share/man/man1installing extension objects: /usr/local/lib/ruby/2.5.0/x86_64-linuxinstalling extension objects: /usr/local/lib/ruby/site_ruby/2.5.0/x86_64-linuxinstalling extension objects: /usr/local/lib/ruby/vendor_ruby/2.5.0/x86_64-linuxinstalling extension headers: /usr/local/include/ruby-2.5.0/x86_64-linuxinstalling extension scripts: /usr/local/lib/ruby/2.5.0installing extension scripts: /usr/local/lib/ruby/site_ruby/2.5.0installing extension scripts: /usr/local/lib/ruby/vendor_ruby/2.5.0installing extension headers: /usr/local/include/ruby-2.5.0/rubyinstalling default gems from lib: /usr/local/lib/ruby/gems/2.5.0 (build_info, cache, doc, extensions, gems, specifications) cmath 1.0.0 csv 1.0.0 fileutils 1.0.2 ipaddr 1.2.0 rdoc 6.0.1 scanf 1.0.0 webrick 1.4.2installing default gems from ext: /usr/local/lib/ruby/gems/2.5.0 (build_info, cache, doc, extensions, gems, specifications) bigdecimal 1.3.4 date 1.0.0 etc 1.0.0 fcntl 1.0.0 fiddle 1.0.0 io-console 0.4.6 json 2.1.0 psych 3.0.2 sdbm 1.0.0 stringio 0.0.1 strscan 1.0.0installing bundled gems: /usr/local/lib/ruby/gems/2.5.0 (build_info, cache, doc, extensions, gems, specifications) minitest 5.10.3 rake 12.3.0 xmlrpc 0.3.0 did_you_mean 1.2.0 net-telnet 0.1.1 power_assert 1.1.1 test-unit 3.2.7installing rdoc: /usr/local/share/ri/2.5.0/systeminstalling capi-docs: /usr/local/share/doc/ruby[root@node01 ruby-2.5.5]#
提示：到此ruby環(huán)境就安裝好了，這里注意一點，yum安裝的ruby版本有點低，所以我們這里選擇編譯安裝；
gem安裝redis包
[root@node01 ruby-2.5.5]# gem install redisERROR: Loading command: install (LoadError) cannot load such file -- zlibERROR: While executing gem ... (NoMethodError) undefined method `invoke_with_build_args' for nil:NilClass[root@node01 ruby-2.5.5]#
提示：gem安裝提示我們沒有zlib，遇到這個問題我們用yum安裝zlib-devel包，然后在把zlib集成到ruby環(huán)境即可；
[root@node01 ruby-2.5.5]# yum install zlib-develLoaded plugins: fastestmirrorLoading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.comResolving Dependencies--> Running transaction check---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed--> Processing Dependency: zlib = 1.2.7-18.el7 for package: zlib-devel-1.2.7-18.el7.x86_64--> Running transaction check---> Package zlib.x86_64 0:1.2.7-17.el7 will be updated---> Package zlib.x86_64 0:1.2.7-18.el7 will be an update--> Finished Dependency ResolutionDependencies Resolved=========================================================================================================== Package Arch Version Repository Size===========================================================================================================Installing: zlib-devel x86_64 1.2.7-18.el7 base 50 kUpdating for dependencies: zlib x86_64 1.2.7-18.el7 base 90 kTransaction Summary===========================================================================================================Install 1 PackageUpgrade ( 1 Dependent package)Total download size: 140 kIs this ok [y/d/N]: yDownloading packages:Delta RPMs disabled because /usr/bin/applydeltarpm not installed.(1/2): zlib-devel-1.2.7-18.el7.x86_64.rpm | 50 kB 00:00:00 /2): zlib-1.2.7-18.el7.x86_64.rpm | 90 kB 00:00:00 -----------------------------------------------------------------------------------------------------------Total 509 kB/s | 140 kB 00:00:00 Running transaction checkRunning transaction testTransaction test succeededRunning transaction Updating : zlib-1.2.7-18.el7.x86_64 1/3 Installing : zlib-devel-1.2.7-18.el7.x86_64 2/3 Cleanup : zlib-1.2.7-17.el7.x86_64 3/3 Verifying : zlib-devel-1.2.7-18.el7.x86_64 1/3 Verifying : zlib-1.2.7-18.el7.x86_64 2/3 Verifying : zlib-1.2.7-17.el7.x86_64 3/3 Installed: zlib-devel.x86_64 0:1.2.7-18.el7 Dependency Updated: zlib.x86_64 0:1.2.7-18.el7 Complete![root@node01 ruby-2.5.5]#
集成zlib庫到Ruby環(huán)境
[root@node01 ruby-2.5.5]# cd ext/zlib/[root@node01 zlib]# ruby extconf.rbchecking for deflateReset() in -lz... yeschecking for zlib.h... yeschecking for crc32_combine() in zlib.h... yeschecking for adler32_combine() in zlib.h... yeschecking for z_crc_t in zlib.h... yescreating Makefile[root@node01 zlib]# make make: *** No rule to make target `/include/ruby.h', needed by `zlib.o'. Stop.[root@node01 zlib]#
提示：遇到這種問題，我們需要打開上面生成的Makefile 把zlib.o: $(top_srcdir)/include/ruby.h替換為zlib.o: ../../include/ruby.h即可
提示：修改好Makefile，再次make
"Makefile" 282L, 8468C written[root@node01 zlib]# makecompiling zlib.clinking shared-object zlib.so[root@node01 zlib]# make install/usr/bin/install -c -m 0755 zlib.so /usr/local/lib/ruby/site_ruby/2.5.0/x86_64-linux[root@node01 zlib]#
再次使用gem安裝redis包
[root@node01 zlib]# gem install redisERROR: While executing gem ... (Gem::Exception) Unable to require openssl, install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources[root@node01 zlib]#
提示：它提示我們?nèi)鄙賝penssl庫，處理方法安裝openssl-devel包，然后把openssl庫集成到ruby環(huán)境中；
安裝openssl-devel包
[root@node01 zlib]# yum install openssl-develLoaded plugins: fastestmirrorLoading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.comResolving Dependencies--> Running transaction check---> Package openssl-devel.x86_64 1:1.0.2k-19.el7 will be installed--> Processing Dependency: openssl-libs(x86-64) = 1:1.0.2k-19.el7 for package: 1:openssl-devel-1.0.2k-19.el7.x86_64--> Processing Dependency: krb5-devel(x86-64) for package: 1:openssl-devel-1.0.2k-19.el7.x86_64--> Running transaction check---> Package krb5-devel.x86_64 0:1.15.1-46.el7 will be installed……省略部分內(nèi)容……Installed: openssl-devel.x86_64 1:1.0.2k-19.el7 Dependency Installed: keyutils-libs-devel.x86_64 0:1.5.8-3.el7 krb5-devel.x86_64 0:1.15.1-46.el7 libcom_err-devel.x86_64 0:1.42.9-17.el7 libkadm5.x86_64 0:1.15.1-46.el7 libselinux-devel.x86_64 0:2.5-15.el7 libsepol-devel.x86_64 0:2.5-10.el7 libverto-devel.x86_64 0:0.2.5-4.el7 pcre-devel.x86_64 0:8.32-17.el7 Dependency Updated: e2fsprogs.x86_64 0:1.42.9-17.el7 e2fsprogs-libs.x86_64 0:1.42.9-17.el7 krb5-libs.x86_64 0:1.15.1-46.el7 libcom_err.x86_64 0:1.42.9-17.el7 libselinux.x86_64 0:2.5-15.el7 libselinux-python.x86_64 0:2.5-15.el7 libselinux-utils.x86_64 0:2.5-15.el7 libsepol.x86_64 0:2.5-10.el7 libss.x86_64 0:1.42.9-17.el7 openssl.x86_64 1:1.0.2k-19.el7 openssl-libs.x86_64 1:1.0.2k-19.el7 Complete![root@node01 zlib]#
集成openssl到ruby環(huán)境中
[root@node01 zlib]# cd ..[root@node01 ext]# lsbigdecimal date fcntl json psych ripper Setup.nt -test-cgi dbm fiber nkf pty rubyvm socket win32configure-ext.mk digest fiddle objspace racc sdbm stringio win32olecontinuation etc gdbm openssl rbconfig Setup strscan zlibcoverage extmk.rb io pathname readline Setup.atheos syslog[root@node01 ext]# cd openssl/[root@node01 openssl]# lsdepend ossl_asn1.c ossl_digest.c ossl_ocsp.c ossl_pkey_rsa.c ossl_x509ext.cdeprecation.rb ossl_asn1.h ossl_digest.h ossl_ocsp.h ossl_rand.c ossl_x509.hextconf.rb ossl_bio.c ossl_engine.c ossl_pkcs12.c ossl_rand.h ossl_x509name.cexts.mk ossl_bio.h ossl_engine.h ossl_pkcs12.h ossl_ssl.c ossl_x509req.cHistory.md ossl_bn.c ossl.h ossl_pkcs7.c ossl_ssl.h ossl_x509revoked.clib ossl_bn.h ossl_hmac.c ossl_pkcs7.h ossl_ssl_session.c ossl_x509store.cMakefile ossl.c ossl_hmac.h ossl_pkey.c ossl_version.h ruby_missing.hmkmf.log ossl_cipher.c ossl_kdf.c ossl_pkey_dh.c ossl_x509attr.copenssl.gemspec ossl_cipher.h ossl_kdf.h ossl_pkey_dsa.c ossl_x509.copenssl_missing.c ossl_config.c ossl_ns_spki.c ossl_pkey_ec.c ossl_x509cert.copenssl_missing.h ossl_config.h ossl_ns_spki.h ossl_pkey.h ossl_x509crl.c[root@node01 openssl]# ruby extconf.rb checking for t_open() in -lnsl... nochecking for socket() in -lsocket... nochecking for openssl/ssl.h... yeschecking for OpenSSL version is 1.0.1 or later... yeschecking for RAND_egd()... yes……省略部分內(nèi)容……checking for X509_get0_notBefore()... nochecking for SSL_SESSION_get_protocol_version()... nochecking for EVP_PBE_scrypt()... nocreating extconf.hcreating Makefile[root@node01 openssl]# makecompiling openssl_missing.cmake: *** No rule to make target `/include/ruby.h', needed by `ossl.o'. Stop.[root@node01 openssl]#
提示：這個錯誤和剛才繼承zlib庫到ruby環(huán)境中類似，處理方式同上面的處理方式一樣修改Makefile文件，在makefile定義變量的位置增加top_srcdir = ../..
再次make
Makefile" 1458L, 49182C written[root@node01 openssl]# makecompiling ossl.ccompiling ossl_asn1.ccompiling ossl_bio.ccompiling ossl_bn.ccompiling ossl_cipher.ccompiling ossl_config.ccompiling ossl_digest.ccompiling ossl_engine.ccompiling ossl_hmac.ccompiling ossl_kdf.ccompiling ossl_ns_spki.ccompiling ossl_ocsp.ccompiling ossl_pkcs12.ccompiling ossl_pkcs7.ccompiling ossl_pkey.ccompiling ossl_pkey_dh.ccompiling ossl_pkey_dsa.ccompiling ossl_pkey_ec.ccompiling ossl_pkey_rsa.ccompiling ossl_rand.ccompiling ossl_ssl.ccompiling ossl_ssl_session.ccompiling ossl_x509.ccompiling ossl_x509attr.ccompiling ossl_x509cert.ccompiling ossl_x509crl.ccompiling ossl_x509ext.ccompiling ossl_x509name.ccompiling ossl_x509req.ccompiling ossl_x509revoked.ccompiling ossl_x509store.clinking shared-object openssl.so[root@node01 openssl]# make install/usr/bin/install -c -m 0755 openssl.so /usr/local/lib/ruby/site_ruby/2.5.0/x86_64-linuxinstalling default openssl libraries[root@node01 openssl]#
再次用gem安裝redis包
[root@node01 openssl]# gem install redisFetching: redis-4.2.1.gem (100%)Successfully installed redis-4.2.1Parsing documentation for redis-4.2.1Installing ri documentation for redis-4.2.1Done installing documentation for redis after 0 seconds1 gem installed[root@node01 openssl]#
提示：到此ruby環(huán)境就準備好了，redis包也安裝好了，接下來我們把redis-trib.rb連接到/usr/bin/下，然后執(zhí)行redis-trib.rb，如果能夠正常執(zhí)行說明ruby環(huán)境沒有問題，如果不能則還需要調(diào)整
軟連接redis-trib.rb到/usr/bin/
[root@node01 openssl]# find /usr/local/src/redis-4.0.9/ -name "redis-trib.rb"/usr/local/src/redis-4.0.9/src/redis-trib.rb[root@node01 openssl]# ln -s /usr/local/src/redis-4.0.9/src/redis-trib.rb /usr/bin/[root@node01 openssl]# redis-trib.rb Usage: redis-trib <command> <options> <arguments ...> create host1:port1 ... hostN:portN --replicas <arg> check host:port info host:port fix host:port --timeout <arg> reshard host:port --from <arg> --to <arg> --slots <arg> --yes --timeout <arg> --pipeline <arg> rebalance host:port --weight <arg> --auto-weights --use-empty-masters --timeout <arg> --simulate --pipeline <arg> --threshold <arg> add-node new_host:new_port existing_host:existing_port --slave --master-id <arg> del-node host:port node_id set-timeout host:port milliseconds call host:port command arg arg .. arg import host:port --from <arg> --copy --replace help (show this help)For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.[root@node01 openssl]#
提示：可以看到redis-trib.rb可以正常打印出幫助信息，說明當前ruby環(huán)境能夠支持redis-trib.rb運行；
添加密碼到剛才下載的redis包的庫文件中
提示：這里需要把對應(yīng)密碼字符串用引號把它引起來；
創(chuàng)建集群
[root@node01 ~]# redis-trib.rb create --replicas 1 192.168.0.41:6379 192.168.0.42:6379 192.168.0.43:6379 192.168.0.41:6380 192.168.0.42:6380 192.168.0.43:6380>>> Creating cluster>>> Performing hash slots allocation on 6 nodes...Using 3 masters:192.168.0.41:6379192.168.0.42:6379192.168.0.43:6379Adding replica 192.168.0.42:6380 to 192.168.0.41:6379Adding replica 192.168.0.43:6380 to 192.168.0.42:6379Adding replica 192.168.0.41:6380 to 192.168.0.43:6379M: 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379 slots:0-5460 (5461 slots) masterM: e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379 slots:5461-10922 (5462 slots) masterM: a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379 slots:10923-16383 (5461 slots) masterS: 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380 replicates a7ace08c36f7d55c4f28463d72865aa1ff74829eS: dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380 replicates 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855S: 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380 replicates e99b0b450e78719d63520cb6efc068d5e8d4d081Can I set the above configuration? (type 'yes' to accept): yes>>> Nodes configuration updated>>> Assign a different config epoch to each node>>> Sending CLUSTER MEET messages to join the clusterWaiting for the cluster to join......>>> Performing Cluster Check (using node 192.168.0.41:6379)M: 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380 slots: (0 slots) slave replicates e99b0b450e78719d63520cb6efc068d5e8d4d081M: e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)M: a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380 slots: (0 slots) slave replicates 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855S: 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380 slots: (0 slots) slave replicates a7ace08c36f7d55c4f28463d72865aa1ff74829e[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@node01 ~]#
提示：從上面的信息可以了解到，我們給指定了6個redis實例的地址和端口，它給我們創(chuàng)建了3個master3個slave，并且在3個master上平均分配了16384個槽位；如果能看到后面的ok all 16384 slots covered,說明集群創(chuàng)建成功；--replicates用來指定集群中每個master的副本數(shù)量，1表示每個master有一個副本；
查看集群狀態(tài)信息
[root@node01 ~]# redis-trib.rb info 192.168.0.41:6379 192.168.0.41:6379 (8c785e6e...) -> 0 keys | 5461 slots | 1 slaves.192.168.0.42:6379 (e99b0b45...) -> 0 keys | 5462 slots | 1 slaves.192.168.0.43:6379 (a7ace08c...) -> 0 keys | 5461 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.[root@node01 ~]# redis-trib.rb check 192.168.0.41:6379 >>> Performing Cluster Check (using node 192.168.0.41:6379)M: 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380 slots: (0 slots) slave replicates e99b0b450e78719d63520cb6efc068d5e8d4d081M: e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)M: a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380 slots: (0 slots) slave replicates 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855S: 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380 slots: (0 slots) slave replicates a7ace08c36f7d55c4f28463d72865aa1ff74829e[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@node01 ~]#
提示：從上面的信息可以看到192.168.0.41/42/43的6379端口都是master，并且都有一個slave節(jié)點；
確認master狀態(tài)
[root@node01 ~]# redis-cli -h 192.168.0.41192.168.0.41:6379> AUTH adminOK192.168.0.41:6379> CLUSTER INFOcluster_state:okcluster_slots_assigned:16384cluster_slots_ok:16384cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:3cluster_current_epoch:6cluster_my_epoch:1cluster_stats_messages_ping_sent:606cluster_stats_messages_pong_sent:586cluster_stats_messages_sent:1192cluster_stats_messages_ping_received:581cluster_stats_messages_pong_received:606cluster_stats_messages_meet_received:5cluster_stats_messages_received:1192192.168.0.41:6379> info replication# Replicationrole:masterconnected_slaves:1slave0:ip=192.168.0.42,port=6380,state=online,offset=854,lag=0master_replid:303943ae1e9f54e3525abda2fd1efff905bf08e3master_replid2:0000000000000000000000000000000000000000master_repl_offset:854second_repl_offset:-1repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:1repl_backlog_histlen:854192.168.0.41:6379> quit[root@node01 ~]# redis-cli -h 192.168.0.42192.168.0.42:6379> auth adminOK192.168.0.42:6379> info replication# Replicationrole:masterconnected_slaves:1slave0:ip=192.168.0.43,port=6380,state=online,offset=896,lag=0master_replid:f4e1b75b8f30be622868814fc4618a4584f3aa32master_replid2:0000000000000000000000000000000000000000master_repl_offset:896second_repl_offset:-1repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:1repl_backlog_histlen:896192.168.0.42:6379> CLUSTER INFOcluster_state:okcluster_slots_assigned:16384cluster_slots_ok:16384cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:3cluster_current_epoch:6cluster_my_epoch:2cluster_stats_messages_ping_sent:662cluster_stats_messages_pong_sent:645cluster_stats_messages_meet_sent:5cluster_stats_messages_sent:1312cluster_stats_messages_ping_received:644cluster_stats_messages_pong_received:667cluster_stats_messages_meet_received:1cluster_stats_messages_received:1312192.168.0.42:6379> quit[root@node01 ~]# redis-cli -h 192.168.0.43192.168.0.43:6379> AUTH adminOK192.168.0.43:6379> CLUSTER INFOcluster_state:okcluster_slots_assigned:16384cluster_slots_ok:16384cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:3cluster_current_epoch:6cluster_my_epoch:3cluster_stats_messages_ping_sent:683cluster_stats_messages_pong_sent:787cluster_stats_messages_meet_sent:4cluster_stats_messages_sent:1474cluster_stats_messages_ping_received:786cluster_stats_messages_pong_received:687cluster_stats_messages_meet_received:1cluster_stats_messages_received:1474192.168.0.43:6379> info replication# Replicationrole:masterconnected_slaves:1slave0:ip=192.168.0.41,port=6380,state=online,offset=952,lag=1master_replid:7cbc7d52452717cd49a0e543f6efdccafe5fe1bdmaster_replid2:0000000000000000000000000000000000000000master_repl_offset:952second_repl_offset:-1repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:1repl_backlog_histlen:952192.168.0.43:6379> quit[root@node01 ~]#
提示：可以看到集群狀態(tài)都是正常ok的，并且41的6379對應(yīng)42的6380，42的6379對應(yīng)43的6380，43的6379對應(yīng)41的6380，剛好master和slave都是錯開的；
查看集群各node對應(yīng)關(guān)系
提示：上面的node對應(yīng)是依據(jù)id號來的。到此redis集群就正常的跑起來了；
驗證：在master上任意寫入一個數(shù)據(jù)看看會發(fā)生什么？
提示：在master01上寫入k1，它提示我們到192.168.0.43：6379上去寫；這是因為我們寫入k1計算的槽位是12706，這個槽位是在master03上，所以在master01上就不能正常的寫入；這同時也告訴我們，要使用redis集群，客戶端還必須能夠智能去理解redis協(xié)議，通過集群返回的信息，把對應(yīng)請求調(diào)度到對應(yīng)master上執(zhí)行；
驗證讀請求
提示：從上面的信息可以確定一點，這三個主節(jié)點他們互相數(shù)據(jù)是完全隔離的，也就說master01不知道m(xù)aster02上到底有沒有對應(yīng)的數(shù)據(jù)，它只知道應(yīng)該去master02上去查看才知道；這就是我們之前說的，每個節(jié)點擁有整個集群的全部狀態(tài)信息（元數(shù)據(jù)），而每個節(jié)點只擁有部分數(shù)據(jù)在本節(jié)點；
驗證：把任意一個master節(jié)點宕機，看看對應(yīng)slave是否能夠被提升為master，接管master呢？
提示：可以看到當master01宕機以后，對應(yīng)slave就被提升為master；
查看master01的配置文件是否有改動？
[root@node01 ~]# cat /usr/local/redis/redis-cluster_6379.conf 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380@16380 master - 0 1596648345000 8 connected 5461-10922dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380@16380 slave 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 0 1596648345510 9 connected62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380@16380 slave a7ace08c36f7d55c4f28463d72865aa1ff74829e 0 1596648343893 4 connectede99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379@16379 slave 91169e71359deed96f8778cf31c823dbd6ded350 0 1596648344903 8 connected8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379@16379 myself,master - 0 1596648345000 9 connected 0-5460a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379@16379 master - 0 1596648345000 3 connected 10923-16383vars currentEpoch 9 lastVoteEpoch 0[root@node01 ~]# ps -ef |grep redisroot 15980 1 0 Aug05 ? 00:00:17 redis-server 0.0.0.0:6380 [cluster]root 30867 1 0 01:19 ? 00:00:02 redis-server 0.0.0.0:6379 [cluster]root 30949 14690 0 01:30 pts/0 00:00:00 grep --color=auto redis[root@node01 ~]# redis-cli -a admin127.0.0.1:6379> info replication# Replicationrole:masterconnected_slaves:1slave0:ip=192.168.0.42,port=6380,state=online,offset=4476,lag=0master_replid:ed690ba2766e6e334ffc298fc6aba8be80465aa8master_replid2:776b5f996d78171e7c5ab5419d65c6c14a253bdamaster_repl_offset:4476second_repl_offset:4071repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:3595repl_backlog_histlen:882127.0.0.1:6379> quit[root@node01 ~]# kill -9 30867[root@node01 ~]# redis-cli -h 192.168.0.42 -p 6380192.168.0.42:6380> AUTH adminOK192.168.0.42:6380> info replication# Replicationrole:masterconnected_slaves:0master_replid:d46c0a3761b61352b1fc75986ab3343e04a89b83master_replid2:ed690ba2766e6e334ffc298fc6aba8be80465aa8master_repl_offset:4518second_repl_offset:4519repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:4071repl_backlog_histlen:448192.168.0.42:6380> quit[root@node01 ~]# cat /usr/local/redis/redis-cluster_6379.conf91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380@16380 master - 0 1596648345000 8 connected 5461-10922dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380@16380 slave 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 0 1596648345510 9 connected62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380@16380 slave a7ace08c36f7d55c4f28463d72865aa1ff74829e 0 1596648343893 4 connectede99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379@16379 slave 91169e71359deed96f8778cf31c823dbd6ded350 0 1596648344903 8 connected8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379@16379 myself,master - 0 1596648345000 9 connected 0-5460a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379@16379 master - 0 1596648345000 3 connected 10923-16383vars currentEpoch 9 lastVoteEpoch 0[root@node01 ~]# grep slaveof /usr/local/redis/6379/etc/redis.conf # Master-Slave replication. Use slaveof to make a Redis instance a copy of# slaveof <masterip> <masterport>[root@node01 ~]#
提示：從上面的信息可以看到，在master01沒有宕機前和宕機以后slave被提升為master后，對應(yīng)的配置信息都沒有發(fā)生變化；我上面為了驗證配置文件是否發(fā)生變化，先是把master01上線，然后把對應(yīng)slave下線在上線，還原master01為master角色；
重新把master上線，看看它的配置文件有什么變化？角色還會是master嗎？
[root@node01 ~]# redis-server /usr/local/redis/6379/etc/redis.conf [root@node01 ~]# ss -tnlState Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 511 *:6379 *:* LISTEN 0 511 *:6380 *:* LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 511 *:16379 *:* LISTEN 0 511 *:16380 *:* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::* [root@node01 ~]# cat /usr/local/redis/redis-cluster_6379.conf 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380@16380 master - 1596649311329 1596649311326 8 connected 5461-109228c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379@16379 myself,slave dbfff4c49a94c0ee55d14401ccc9245af3655427 0 1596649311326 9 connectede99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379@16379 slave 91169e71359deed96f8778cf31c823dbd6ded350 0 1596649311331 8 connecteda7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379@16379 master - 0 1596649311332 3 connected 10923-1638362ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380@16380 slave a7ace08c36f7d55c4f28463d72865aa1ff74829e 0 1596649311331 4 connecteddbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380@16380 master - 0 1596649311332 10 connected 0-5460vars currentEpoch 10 lastVoteEpoch 0[root@node01 ~]# redis-cli 127.0.0.1:6379> AUTH adminOK127.0.0.1:6379> info replication# Replicationrole:slavemaster_host:192.168.0.42master_port:6380master_link_status:upmaster_last_io_seconds_ago:5master_sync_in_progress:0slave_repl_offset:4686slave_priority:100slave_read_only:1connected_slaves:0master_replid:d46c0a3761b61352b1fc75986ab3343e04a89b83master_replid2:0000000000000000000000000000000000000000master_repl_offset:4686second_repl_offset:-1repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:4519repl_backlog_histlen:168127.0.0.1:6379>
提示：可以看到，重新把master01上線以后，它自動降級為slave了，并且配置文件也從master需改成slave了；這應(yīng)該是redis cluster在檢測到192.168.0.41：6379上線后，觸發(fā)把內(nèi)存中的配置同步到對應(yīng)文件，使得對應(yīng)配置文件也被修改了；到此一個redis cluster 就搭建測試完成了；
