Redis cluster setup full record

Updated: September 19, 2017 08:47:07 by Ouka Fu

This article summarizes the basic knowledge of the concept of redis cluster, as well as the problems encountered by individuals in the construction of redis cluster and solutions, very detailed, small partners in need can refer to it

A Redis cluster is an assembly that provides data sharing among multiple Redis nodes.

Commands that handle multiple keys are not supported in the Redis cluster.

Redis clusters provide a degree of availability through partitioning. You can continue to process commands when a node is down or unavailable.

Redis cluster data fragmentation

In a Redis cluster, which uses sharding instead of consistency hashing, a Redis cluster contains 16,384 hash slots, and each key in the database has one of these hash slots. After CRC16 verification, 16384 is modelled to determine.

Adding says there are now three nodes in the cluster, so

① Node A contains hash slots 0 to 5500.

Node B contains hash slots 5501 through 11000.

③ Node C contains hash slots 11001 through 16384.

And then if we want to add a node, we will get some slots from each of the three nodes ABC to D. If we remove a node, we will move the separating slot of the removed node above the remaining slot.

Redis master/slave replication model

To achieve high availability of Redis cluster. Redis provides a master-slave replication model. Each node will have N-1 copies.

If we add a slave node to each node when creating the cluster, if one of the master nodes fails, the slave node will be used as the new master node and continue to provide services. However, if both the master and slave nodes are lost, they can no longer be used.

Redis cluster construction

A Redis cluster consists of multiple cloud new Redis instances in cluster mode. The cluster mode of the instance needs to be enabled by configuration.

Here is an example of a cluster profile with minimal options:

port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
pidfile /var/run/redis_xxxx.pid
dir /usr/local/redis-cluster/xxxx

port Specifies the port number of the node

The cluster-enabled yes option enables the cluster mode for the instance

Cluster-config-file nodes.conf sets the path to the configuration file for saving nodes. This file is created when the cluster is started without modification.

cluster-node-timeout 5000: specifies the waiting time for mode failure. That is, if the node is not accessed within 5 seconds, it is considered unavailable.

appendonly yes: used to enable aof persistence

pidfile /var/run/redis_xxxx.pid Specifies the location of the pidfile. xxxx is the port number

dir /usr/local/redis-cluster/xxxx Sets the working directory, where xxxx is the port number.

The files in the directory at this point look like this

[root@localhost 10:49 /usr/local/redis-cluster]# ll 7000 7001 7002 7003 7004 7005 7000: Total Usage 7080-rw-r --r--. 1 root root 57787 September 10 10:44 redis.conf-rwxr-xr-x. 1 root root 7185872 September 10 09:50 redis-server 7001: Total Usage 7080-rw-r --r--. 1 root root 57787 September 10 10:46 redis.conf-rwxr-xr-x. 1 root root 7185872 September 10 10:04 redis-server 7002: Total Usage 7080-rw-r --r--. 1 root root 57787 September 10 10:46 redis.conf-rwxr-xr-x. 1 root root 7185872 September 10 10:04 redis-server 7003: Total Usage 7080-rw-r --r--. 1 root root 57787 September 10 10:47 redis.conf-rwxr-xr-x. 1 root root 7185872 September 10 10:04 redis-server 7004: Total Usage 7080-rw-r --r--. 1 root root 57787 September 10 10:48 redis.conf-rwxr-xr-x. 1 root root 7185872 September 10 10:04 redis-server 7005: Total Usage 7080-rw-r --r--. 1 root root 57787 September 10 10:48 redis.conf-rwxr-xr-x. 1 root root 7185872 September 10 10:04 redis-server [root@localhost 10:49 /usr/local/redis-cluster]#

Let's start the six nodes

[root@localhost 10:51 /usr/local/redis-cluster]#./7000/ redis-server. /7000/redis.conf 3547:C 10 Sep 10:51:58.519 # oO0OoO0OoO0Oo Redis is starting oo0ooo0ooo0ooo0oo 3547:C 10 Sep 10:51:58.519 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3547, just started 3547:C 10 Sep 10:51:58.519 # Configuration loaded [root@localhost 10:51 /usr/local/redis-cluster]# /7001/ Redis-server./7001/ Redis.conf 3552:C 10 Sep 10:52:05.549 # oO0OoO0OoO0Oo Redis is starting oo0ooo0ooo0ooo0oo 3552:C 10 Sep 10:52:05.550 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3552, just started 3552:C 10 Sep 10:52:05.550 # Configuration loaded [root@localhost 10:52 /usr/local/redis-cluster]# ./7002/ Redis-server./7002/redis.conf 3557:C 10 Sep 10:52:13.098 # oO0OoO0OoO0Oo Redis is starting oo0ooo0ooo0ooo0oo 3557:C 10 Sep 10:52:13.098 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3557, just started 3557:C 10 Sep 10:52:13.098 # Configuration loaded [root@localhost 10:52 /usr/local/redis-cluster]# ./7003/ Redis-server./7003/redis.conf 3563:C 10 Sep 10:52:18.986 # oO0OoO0OoO0Oo Redis is starting oo0ooo0ooo0ooo0oo 3563:C 10 Sep 10:52:18.986 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3563, just started 3563:C 10 Sep 10:52:18.986 # Configuration loaded [root@localhost 10:52 /usr/local/redis-cluster]# ./7004/ Redis-server./7004/redis.conf 3568:C 10 Sep 10:52:23.709 # oO0OoO0OoO0Oo Redis is starting oo0ooo0ooo0ooo0oo 3568:C 10 Sep 10:52:23.710 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3568, just started 3568:C 10 Sep 10:52:23.710 # Configuration loaded [root@localhost 10:52 /usr/local/redis-cluster]# ./7005/ Redis-server./7005/redis.conf 3573:C 10 Sep 10:52:27.146 # oO0OoO0OoO0Oo Redis is starting oo0ooo0ooo0ooo0oo 3573:C 10 Sep 10:52:27.147 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3573, just started 3573:C 10 Sep 10:52:27.147 # Configuration loaded [root@localhost 10:52 /usr/local/redis-cluster]#

To view the process after startup:

[root@localhost 10:52 /usr/local/redis-cluster]# ps aux|grep redis root 3548 0.2 0.2 41964 2288? Ssl 10:51 07:00. /7000/redis-server 127.0.0.1:7000 [cluster] root 3553 0.1 0.2 41964 2288? Ssl 10:52 0:00./7001/redis-server 127.0.0.1:7001 [cluster] root 3558 0.2 0.2 41964 2288? Ssl 10:52 07:00. /7002/redis-server 127.0.0.1:7002 [cluster] root 3564 0.1 0.2 41964 2292? Ssl 10:52 07:00. /7003/redis-server 127.0.0.1:7003 [cluster] root 3569 0.2 0.2 41964 2292? Ssl 10:52 07:00. /7004/redis-server 127.0.0.1:7004 [cluster] root 3574 0.1 0.2 41964 2288? Ssl 10:52 0:00./7005/ Redis-server 127.0.0.1:7005 [cluster] root 3580 0.0 0.0 6048 784 pts/2 S+ 10:52 0:00 grep redis [root@localhost 10:52 /usr/local/redis-cluster]#

Look again at the file directory:

[root@localhost 11:01 /usr/local/redis-cluster]# ll 7000 7001 7002 7003 7004 7005 7000: Total Usage 7084-rw-r --r--. 1 root root 0 September 10 10:51 appendonly.aof -rw-r--r--. 1 root root 114 September 10 10:51 nodes-70.conf -rw-r--r-- 1 root root 57787 Sep 10 10:44 redis. conf-rwxr-xr-x. 1 root root 7185872 Sep 10 09:50 redis-server 7001: Total Usage 7084-rw-r --r--. 1 root root 0 September 10 10:52 appendonly.aof -rw-r--r--. 1 root root 114 September 10 10:52 nodes-701.conf -rw-r--r-- 1 root root 57787 Sep 10 10:46 redis. conf-rwxr-xr-x. 1 root root 7185872 Sep 10 10:04 redis-server 7002: Total Usage 7084-rw-r --r-- 1 root root 0 September 10 10:52 appendonly.aof -rw-r--r-- 1 root root 114 September 10 10:52 nodes-7002.conf -rw-r--r-- 1 root root 57787 Sep 10 10:46 redis. conf-rwxr-xr-x. 1 root root 7185872 Sep 10 10:04 redis-server 7003: Total Usage 7084-rw-r --r-- 1 root root 0 September 10 10:52 appendonly.aof -rw-r--r-- 1 root root 114 September 10 10:52 nodes-7003.conf -rw-r--r-- 1 root root 57787 Sep 10 10:47 redis. conf-rwxr-xr-x. 1 root root 7185872 Sep 10 10:04 redis-server 7004: Total Usage 7084-rw-r --r-- 1 root root 0 September 10 10:52 appendonly.aof -rw-r--r-- 1 root root 114 September 10 10:52 nodes-7004.conf -rw-r--r-- 1 root root 57787 Sep 10 10:48 redis. conf-rwxr-xr-x. 1 root root 7185872 Sep 10 10:04 redis-server 7005: Total Usage 7084-rw-r --r-- 1 root root 0 September 10 10:52 appendonly.aof -rw-r--r-- 1 root root 114 September 10 10:52 nodes-7005.conf -rw-r--r-- 1 root root 57787 September 10 10:48 redis.conf -rwxr-xr-x. 1 root root 7185872 September 10 10:04 redis-server [root@localhost 11:01 /usr/local/redis-cluster]#

You can see that aof and nodes files are generated, and dump files are also generated when saving.

Looking at pid

[root@localhost 11:03 /usr/local/redis-cluster]# ll /var/run/redis* -rw-r--r-- 1 root root 5 September 10 at 10:51 /var/run/redis_7000. pid-rw -r--r-- 1 root root 5 September 10 10:52 /var/run/redis_7001. pid-rw -r--r-- 1 root root 5 September 10 10:52 /var/run/redis_7002. pid-rw -r--r-- 1 root root 5 September 10 10:52 /var/run/redis_7003. pid-rw -r--r-- 1 root root 5 September pid -rw-r--r--. 1 root root 5 September 10:52 /var/run/redis_7005.pid [root@localhost 11:03 /usr/local/redis-cluster]#

Next, we need to use the redis-trib tool and the six nodes to create the cluster.

Redis-trib is located in the src directory of the redis source code.

Let's copy it to the redis-cluster directory.

[root@localhost 11:03 /usr/local/redis-cluster]# ll .. /redis/src/redis-trib* -rwxrwxr-x. 1 root root 60843 July 24 22:58... /redis/src/redis-trib.rb [root@localhost 11:04 /usr/local/redis-cluster]# cp .. /redis/src/ redis-trib.rb. / [root@localhost 11:05 /usr/local/redis-cluster]# ll Total usage 84 drwxr-xr-x.2 root root 4096 September 10 10:51 7000 DRWXr-Xr-X-2 root root 4096 September 10 10:52 7001 DRWXR-XR-X-2 root root 4096 September 10 10:52 7002 DRWXR-XR-X-2 root root 4096 September 10 10:52 7003 drwxr-xr-x. 2 root root 4096 September 10 10:52 7004 drwxr-xr-x. 2 root root 4096 September 10 10:52 7005-rwxr-xr-x.1 root root 60843 September 10 11:05 redis-trib.rb [root@localhost 11:05 /usr/local/redis-cluster]#

To start the cluster:

[root@localhost 11:13 /usr/local/redis-cluster]#./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 /usr/bin/env: ruby: There is no file or directory [root@localhost 11:15 /usr/local/redis-cluster]#

Below is my failed installation version, followed by a successful version.

Because it's a ruby program, we need to install ruby,

yum -y install ruby

To continue starting the cluster:

[root@localhost 11:21 /usr/local/redis-cluster]#./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005./redis-trib.rb:24:in 'require': no such file to load -- rubygems (LoadError) from ./redis-trib.rb:24 [root@localhost 11:22 /usr/local/redis-cluster]#

We still get an error saying we need rubygems, so let's go ahead and install:

yum -y install rubygems

To continue starting the cluster:

[root@localhost 11:24 /usr/local/redis-cluster]#./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 /usr/lib/ruby/site_rubygems/custom_requirement. rb:31:in  `gem_original_require': no such file to load -- redis (LoadError) from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in 'require' from ./redis-trib.rb:25 [root@localhost 11:24 /usr/local/redis-cluster]#

Still reporting an error... We need to install a later version of ruby.

So, I decided to uninstall 1.8 ruby first.

yum -y remove ruby

Then reinstall

Wget https://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.1.tar.gz tar XZVF ruby - against 2.4.1. Tar. Gz CD ruby - against 2.4.1. / configure make make install

Then check out the version of ruby:

[root@localhost 12:47 ~]# ruby -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [i686-linux]
[root@localhost 12:47 ~]#

Finally, in order to run Redis-trib we need to install the redis gem.

If the following error occurs

[root@localhost 13:01 /usr/local/redis-cluster]# gem install redis
ERROR: Loading command: install (LoadError)
    cannot load such file -- zlib
ERROR: While executing gem ... (NoMethodError)
  undefined method `invoke_with_build_args' for nil:NilClass

We continue to open the cluster: still reported an error, after several toss, still not solved

Best of all, find the following method.

Successful version

Install ruby using rvm, the ruby administration tool.

install rvm first, and if curl is not found in the following command prompt, you can install crul by running yum install -y crul.

[root@localhost 19:45 ~]# curl -L get.rvm.io | bash -s stable % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 24090 100 24090 0 0 11517 0 0:00:02 0:00:02 --:--:-- 48276 Downloading Downloading https://github.com/rvm/rvm/archive/1.29.3.tar.gz GPG: https://github.com/rvm/rvm/releases/download/1.29.3/1.29.3.tar.gz.asc Signature created by JST on Monday September 11, 2017 05:59:21, using RSA, key number BF04FF17 gpg: Signature cannot be checked: No public key Warning, RVM 1.26.0 introduces signed releases and automated check of signatures when GPG software found. Assuming you trust Michal Papis import the mpapis public key (downloading the signatures). GPG signature verification failed for '/ usr/local/RVM/archives/RVM - 1.29.3. TGZ' - 'https://github.com/rvm/rvm/releases/download/1.29.3/1.29.3.tar.gz.asc'! Try to install GPG v2 and then fetch the public key: gpg2 --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 or if it fails: command curl -sSL https://rvm.io/mpapis.asc | gpg2 --import - the key can be compared with: https://rvm.io/mpapis.asc https://keybase.io/mpapis NOTE: GPG version 2.1.17 have a bug which cause failures during fetching keys from remote server. Please downgrade or upgrade to newer version (if available) or use the second method described above. [root@localhost 19:45 ~]#

Finding it failed, we followed the prompts.

[root@localhost 19:45 ~]# curl -sSL https://rvm.io/mpapis.asc | gpg2 --import - gpg: Key ring '/root/.gnupg/secring.gpg' Established gpg: /root/.gnupg/trustdb.gpg: established trust database gpg: key D39DC0E3: Public key Michal Papis (RVM signing) <mpapis@gmail.com> Imported gpg: Total number of processing: 1 gpg: imported: 1 (RSA: 1) gpg: No absolutely trusted keys were found. [root@localhost 19:45 ~]# curl -L get.rvm.io | bash -s stable % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 24090 100 24090 0 0 3763 0 0:00:06 0:00:06 --:--:-- 3763 Downloading https://github.com/rvm/rvm/archive/1.29.3.tar.gz Downloading GPG: https://github.com/rvm/rvm/releases/download/1.29.3/1.29.3.tar.gz.asc Signature created by JST on Monday September 11, 2017 at 05:59:21, using RSA, key number BF04FF17 gpg: Complete signature, from "Michal Papis (RVM signing) <mpapis@gmail.com>" gpg: i.e. "Michal Papis <michal.papis@toptal.com>" gpg: "[jpeg image of size 5015]" gpg: Warning: This key is not certified with a trusted signature! gpg: There is no evidence that this signature belongs to its purported holder. Main key fingerprint: 409B 6B17 96C2 7546 2A17 0311 3804 BB82 D39D C0E3 Child key fingerprint: 166 b 62 c9 E5F4 DA30 0 d94 AC36 E206 C29F BF04 FF17 GPG verified '/ usr/local/RVM/archives/RVM - 1.29.3. TGZ' Creating group 'rvm' Installing RVM to /usr/local/rvm/ Installation of RVM in /usr/local/rvm/ is almost complete: * First you need to add all users that will be using rvm to 'rvm' group, and logout - login again, anyone using rvm will be operating with `umask u=rwx,g=rwx,o=rx`. * To start using RVM you need to run `source /etc/profile.d/rvm.sh` in all your open shell windows, in rare cases you need to reopen all shell windows. [root@localhost 19:46 ~]

We find that rvm has been installed successfully. In order for the configuration to take effect immediately, we need to import the rvm configuration file with the source command.

Check the path of the configuration file and import it.

[root@localhost 19:46 ~]# find / -name rvm.sh
/etc/profile.d/rvm.sh
[root@localhost 19:49 ~]# source /etc/profile.d/rvm.sh

Next, you can install a higher version of ruby, it is recommended to install version 2.2.3, otherwise, when gen install redis, you may report the following error:

[root@localhost 19:59 ~]# gem install redis Fetching: Redis-4.0.0. gem (100%) ERROR: Error installing redis: redis requires Ruby version >= 2.2.2.

Without further ado, install ruby version 2.2.3

[root@localhost 20:16 ~]# rvm install 2.3.3 Searching for binary rubies, this might take some time. No binary rubies available for: centos/6/i386/ruby-2.3.3. Continuing with compilation. Please read 'rvm help mount' to get more information on binary rubies. Checking requirements for centos. Requirements installation successful. Installing Ruby from source to: / usr/local/RVM/rubieslast/ruby - 2.3.3, this may take a while depending on your CPU (s)... ruby-2.3.3 - #downloading ruby-2.3.3, this may take a while depending on your connection... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 13.7M. % Total % Received % Xferd Average speed time time current dload upload total spent left speed 100 13.7M 00 1285k 0 0:00:10 0:00:10 --:--:-- 1394k ruby-2.3.3 - #extracting ruby-2.3.3 to /usr/local/rvm/src/ruby-2.3.3.... Ruby - 2.3.3 - # applying patch/usr/local/RVM/patches/ruby/ruby_2_3_gcc7 patch. The ruby - 2.3.3 - # applying patch / usr/local/RVM/patches/ruby / 2.3.3 / random_c_using_NR_prefix patch. Ruby -- 2.3.3 #configuring.......................................................... ruby-2.3.3 - #post-configuration... Ruby - 2.3.3 - # compiling... Ruby - 2.3.3 - # installing... ruby-2.3.3 - #making binaries executable... ruby-2.3.3 - #downloading rubygems-2.6.13 ruby-2.3.3 - #extracting rubygems-2.6.13..... ruby-2.3.3 - #removing old rubygems......... Ruby - 2.3.3 - # installing rubygems - 2.6.13... ruby-2.3.3 - #gemset created /usr/local/rv/gems /ruby-2.3.3@global ruby-2.3.3 - #importing gemset /usr/local/rvm/gemsets/global.gems.......................................... | ruby-2.3.3 - #generating global wrappers........ Ruby-2.3.3 - #gemset created /usr/local/rv/gems /ruby-2.3.3 ruby-2.3.3 - #importing gemsetfile / usr/local/RVM gemsets/default. The gems evaluated to empty ruby gem list - 2.3.3 - # generating default wrappers... ruby-2.3.3 - #adjusting #shebangs for (gem irb erb ri rdoc testrb rake). Install of ruby-2.3.3 - #complete Please be aware that you just installed a ruby that requires 2 patches just to be compiled on an up to date linux system. This may have known and unaccounted for security vulnerabilities. Please consider upgrading to ruby-2.4.1 which will have all of  the latest security patches. Ruby was built without documentation, to build it run: rvm docs generate-ri [root@localhost 20:27 ~]# ruby -v ruby 2.3.3p222 (2016-11-21 revision 56859) [i686-linux] [root@localhost 20:28 ~]# gem-v 2.6.13 [root@localhost 20:28 ~]#

As you can see, not only is ruby 2.3.3 installed, but rubygems is also installed.

If you want to remove a version of ruby, use this command: rvm remove 1.9.3.

We continue to install:

[root@localhost 20:28 ~]# gem install redis Fetching: Redis-4.0.0. gem (100%) Successfully installed redis-4.0.0 Parsing documentation for redis-4.0.0 Installing ri documentation for redis-4.0.0 Done installing documentation for redis after 3 seconds 1 gem installed [root@localhost 20:33 ~]#

Here it is. It took two nights. The next step is to start the cluster using our redis-trib.

For convenience, I made a copy of the redis-trib.rb file into the /usr/local/bin directory, because this directory is under PATH, and the commands in it can be executed directly.

Reconfirm that the next six redis nodes are running ok.

[root@localhost 20:35 /usr/local/redis-cluster]# ps aux|grep redis root 2486 0.1 0.2 46060 2304? Ssl 18:30 0:12./7000/redis-server 127.0.0.1:7000 [cluster] root 2491 0.1 0.2 46060 2308? Ssl 18:30 0:13./7001/redis-server 127.0.0.1:7001 [cluster] root 2496 0.1 0.2 46060 2308? Ssl 18:30 0:13./7002/redis-server 127.0.0.1:7002 [cluster] root 2501 0.1 0.2 46060 2304? Ssl 18:30 0:13. /7003/redis-server 127.0.0.1:7003 [cluster] root 2506 0.1 0.2 46060 2308? Ssl 18:30 0:14./7004/redis-server 127.0.0.1:7004 [cluster] root 2511 0.1 0.2 46060 2304? Ssl 18:30 0:12./7005/ Redis-server 127.0.0.1:7005 [cluster] root 31426 0.0 0.0 6048 784 pts/2 S+ 20:35 0:00 grep redis [root@localhost 20:35 /usr/local/redis-cluster]#

[root@localhost 20:35 /usr/local/redis-cluster]# redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 >>> Creating cluster >>> Performing hash slots allocation on  6 nodes... Using 3 masters: 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 Adding replica 127.0.0.1:7003 to 127.0.0.1:7000 Adding replica 127.0.0.1:7004 to 127.0.0.1:7001 Adding replica 127.0.0.1:7005 to 127.0.0.1:7002 M: 127.0.0.1:7004 to 127.0.0.1:7001 Adding replica 127.0.0.1:7005 to 127.0.0.1:7002 m: 6 d996f9e34f40b02afe06aa9d3c8f18a41875cb2 127.0.0.1:7000 slots: 0-5460 (5461 slots) master M: 9 bf2ee5d32fc350ec411d3eaad18f82492796e99 127.0.0.1:7001 slots: 5461-10922 (5462 slots) master M: 6 de604de12b4b4a3be46766bb95ccf4618dc0d75 127.0.0.1:7002 slots: 10923-16383 (5461 slots) master S: F1bdbd841e37825169426486d6502e2cd99b76fe 127.0.0.1:7003 6 replicates d996f9e34f40b02afe06aa9d3c8f18a41875cb2 S: 3 ff95d180ec47fc58c9c5f73e2f54354128bfe7e 127.0.0.1:7004 replicates 9 bf2ee5d32fc350ec411d3eaad18f82492796e99 S: A4a09d80a4bb6c82345c376c59e5ae49e1d49701 127.0.0.1:7005 6 replicates de604de12b4b4a3be46766bb95ccf4618dc0d75 Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join... >>> Performing Cluster Check (using node 127.0.0.1:7000) M: 6 d996f9e34f40b02afe06aa9d3c8f18a41875cb2 127.0.0.1:7000 slots: 0-5460 (5461 slots) master 1 additional up (s) s: F1bdbd841e37825169426486d6502e2cd99b76fe 127.0.0.1:7003 slots: (0 slots) slave replicates 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2 M: 6 de604de12b4b4a3be46766bb95ccf4618dc0d75 127.0.0.1:7002 slots: 10923-16383 (5461 slots) master 1 additional up (s) M: 9 bf2ee5d32fc350ec411d3eaad18f82492796e99 127.0.0.1:7001 slots: 5461-10922 (5462 slots) master 1 additional up (s) s: 3 ff95d180ec47fc58c9c5f73e2f54354128bfe7e 127.0.0.1:7004 slots: (0 slots) slave replicates 9bf2ee5d32fc350ec411d3eaad18f82492796e99 S: A4a09d80a4bb6c82345c376c59e5ae49e1d49701 127.0.0.1:7005 slots: (0 slots) slave replicates 6de604de12b4b4a3be46766bb95ccf4618dc0d75 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. [root@localhost 20:37 /usr/local/redis-cluster]#

During the creation process, redis will give us a preview of the configuration, make sure that it is OK, type yes and press enter. The system creates a cluster structure of three masters and three slaves for us.

You can see All 16384 slots covered.

redis-cli

redis-cli By default, you connect to the local redis server on port 6379.

There are now six ports, so you need to specify parameters.

redis-cli -h xxx -p xxx -a xxx

-h: specifies the server

-p: specifies the port number

-a: Specifies the password

-c: Enables the cluster mode

[root@localhost 20:53 /usr/local/redis-cluster]# redis-cli -c -p 7000 127.0.0.1:7000> set name zhangsan -> Redirected to slot [5798] located at 127.0.0.1:7001 OK 127.0.0.1:7001> exit [root@localhost 20:53 /usr/local/redis-cluster]# redis-cli -c -p 7002 127.0.0.1:7002> get name -> Redirected to slot [5798] located at 127.0.0.1:7001 "zhangsan" 127.0.0.1:7001 >

As you can see, you can already share the data.

But what if we kill one of the master nodes?

[root@localhost 19:52 /usr/local/redis-cluster]# redis-cli -p 7002 debug segfault Error: Server closed the connection [root@localhost 19:53 /usr/local/redis-cluster]# redis-trib.rb check 127.0.0.1:7000 >>> Performing Cluster Check (using node 127.0.0.1:7000) M: 6 d996f9e34f40b02afe06aa9d3c8f18a41875cb2 127.0.0.1:7000 slots: 0-5460 (5461 slots) master 1 additional up (s) M: 9 bf2ee5d32fc350ec411d3eaad18f82492796e99 127.0.0.1:7001 slots: 5461-10922 (5462 slots) master 1 additional up (s) s: 3 ff95d180ec47fc58c9c5f73e2f54354128bfe7e 127.0.0.1:7004 slots: (0 slots) slave replicates 9bf2ee5d32fc350ec411d3eaad18f82492796e99 M: A4a09d80a4bb6c82345c376c59e5ae49e1d49701 127.0.0.1:7005 slots: 10923-16383 (5461 slots) master 0 additional up (s) s: F1bdbd841e37825169426486d6502e2cd99b76fe 127.0.0.1:7003 slots: (0 slots) slave replicates 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. [root@localhost 19:54 /usr/local/redis-cluster]#

You can see that the node with port 7005 has become the primary node. What if we go ahead and take down node 7005?

[root@localhost 19:58 /usr/local/redis-cluster]# redis-cli -p 7005 debug segfault Error: Server closed the connection [root@localhost 19:58 /usr/local/redis-cluster]# redis-trib.rb check 127.0.0.1:7000 [ERR] Sorry, can't connect to node 127.0.0.1:7005 >>> Performing Cluster Check (using node 127.0.0.1:7000) M: 6 d996f9e34f40b02afe06aa9d3c8f18a41875cb2 127.0.0.1:7000 slots: 0-5460 (5461 slots) master 1 additional up (s) M: 9 bf2ee5d32fc350ec411d3eaad18f82492796e99 127.0.0.1:7001 slots: 5461-10922 (5462 slots) master 1 additional up (s) s: 3 ff95d180ec47fc58c9c5f73e2f54354128bfe7e 127.0.0.1:7004 slots: (0 slots) slave replicates 9bf2ee5d32fc350ec411d3eaad18f82492796e99 S: F1bdbd841e37825169426486d6502e2cd99b76fe 127.0.0.1:7003 slots: (0 slots) slave replicates 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [ERR] Not all 16384 slots are covered by nodes. [root@localhost 19:58 /usr/local/redis-cluster]#

You can see that the cluster has problems.

In Memory:

The following problems occur when building a multi-machine multipoint cluster:

1. Been waiting...

>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.......................

Workaround: Verify that the port is open. If it is not developed, you can modify the /etc/sysconfig/iptables file, open the ports (copy the previous line of open ports to modify), and then service iptables restart to restart the firewall.

Alternatively, the service iptables stop can be used to disable the firewall

2. The slot is used

/ usr/local/RVM/gems/ruby - 2.3.3 / gems/redis - 4.0.0 / lib/redis/client. The rb: 119: in ` call ': ERR Slot 5461 is already busy (Redis::CommandError) from / usr/local/RVM/gems/ruby - 2.3.3 / gems/redis - 4.0.0 / lib/redis rb: 2742: in ` block method_missing 'in the from /usr/local/rv/gems /ruby-2.3.3/gems/redis-4.0.0/lib/redis.rb:45:in 'block in synchronize' from / usr/local/RVM/rubieslast/ruby - 2.3.3 / lib/ruby / 2.3.0 / monitor. The rb: 214: in ` mon_synchronize 'the from /usr/local/rv/gems /ruby-2.3.3/gems/redis-4.0.0/lib/redis.rb:45:in 'synchronize' from / usr/local/RVM/gems/ruby - 2.3.3 / gems/redis - 4.0.0 / lib/redis rb: 2741: in ` method_missing 'the from /usr/local/bin/redis-trib.rb:212:in `flush_node_config' from /usr/local/bin/redis-trib.rb:776:in `block in flush_nodes_config' from /usr/local/bin/redis-trib.rb:775:in `each' from /usr/local/bin/redis-trib.rb:775:in `flush_nodes_config' from /usr/local/bin/redis-trib.rb:1296:in `create_cluster_cmd' from /usr/local/bin/redis-trib.rb:1700:in `<main>'

Workaround: Delete several files in the dir directory of redis.conf except the redis.conf file.

If you still have the same problem, consider executing the following two commands on all nodes.

redis-cli -p port number -h physical address of the host FLUSHALL

redis-cli -p port number -h host physical address CLUSTER RESET SOFT

Articles you may be interested in:

Ruby basics and the like

This article mainly introduces the basic knowledge of Ruby, such as this article explains the creation of classes, accessors, class variables, constants, class methods, method access, factory methods, module module and other content, need friends can refer to the next
2015-04-04
Learning notes for Range objects in Ruby

This article mainly introduces the Ruby Range object learning notes, this article explains the definition of Range object, some of the use of Range object skills and other content, need friends can refer to
2014-11-11
Summary of the basic operations of the Hash structure in Ruby

Hash is a key and value corresponding data structure,Ruby directly with the Hash class to provide support for it, here we collate the basic operation methods of the Hash structure in Ruby summary, first to review the basic knowledge of Hash:
2016-06-06
Summary of the differences between public, private, and protected in Ruby

This article mainly introduces the difference between public, private and protected in Ruby, that is, the difference between access control characters in Ruby, and the need for friends can refer to it
2014-08-08
A few programming tips to speed up Ruby on Rails

This article mainly introduces several accelerated Ruby on Rails programming skills, this article comes from the IBM official website technical documentation, the need for friends can refer to
2015-04-04
Talk about Strong Parameters mechanism in Rails 4

In this paper, the Parameters in Rails3 are compared with the newly introduced Strong Parameters mechanism in Rails4, so as to obtain their direct similarities and differences
2014-06-06
rudy overloading method details

rudy Overload method details...
2007-11-11
6 methods for executing shell commands in Ruby

This article mainly introduces the Ruby call execution shell command 6 methods, this article lists Ruby can call and execute the Linux system Shell command 6 methods, need friends can refer to
2014-10-10
Similarities and differences between module_function and extend self in Ruby

This article mainly introduces the common points and differences between module_function and extend self in Ruby. It is very detailed and practical to facilitate a better understanding of module_function and extend self
2017-05-05
Summary of the basic directory structure of applications built by Ruby on Rails

Ruby on Rails is the largest Web development framework in the Ruby world. To master the construction of Rails programs, an understanding of its directory structure is necessary. Here is a summary of the basic directory structure of applications built by Ruby on Rails
2016-05-05