MC Scylla Cluster

Create a ScyllaDB Cluster - Single Data Center (DC)

Prerequisites

  • Make sure that all the ports are open.

    Port Description Protocol
    9042 CQL (native_transport_port) TCP
    9142 SSL CQL (secure client to node) TCP
    7000 Inter-node communication (RPC) TCP
    7001 SSL inter-node communication (RPC) TCP
    7199 JMX management TCP
    10000 Scylla REST API TCP
    9180 Prometheus API TCP
    9100 node_exporter (Optionally) TCP
    9160 Scylla client port (Thrift) TCP
    19042 Native shard-aware transport port TCP
    19142 Native shard-aware transport port (SSL) TCP
  • Obtain the IP addresses of all nodes which have been created for the cluster.

  • Select a unique name as cluster_name for the cluster (identical for all the nodes in the cluster).

Procedure

These steps need to be done for each of the nodes in the new cluster.

  1. Install Scylla on a node.Follow the Scylla install procedure up to scylla.yaml configuration phase.

    Note: In case that the node starts during the process follow these instructions

    Procedure

    1. Stop the Scylla service
    sudo systemctl stop scylla-server
    
    1. Delete the Data and Commitlog folders.
    sudo rm -rf /data/ds/sdc/scylla/data
    sudo find /data/ds/sdc/scylla/commitlog -type f -delete
    sudo find /data/ds/sdc/scylla/hints -type f -delete
    sudo find /data/ds/sdc/scylla/view_hints -type f -delete
    
  2. In the scylla.yaml file, edit the parameters listed below. The file can be found under /etc/scylla/

    • cluster_name - Set the selected cluster_name
    • seeds - Specify the IP of the first node and only the first node. New nodes will use the IP of this seed node to connect to the cluster and learn the cluster topology and state.
    • listen_address - IP address that Scylla used to connect to other Scylla nodes in the cluster
    • endpoint_snitch - Set the selected snitch
    • rpc_address - Address for client connection (Thrift, CQL)
  3. This step needs to be done only if you are using the GossipingPropertyFileSnitch. If not, skip this step. In the cassandra-rackdc.properties file, edit the parameters listed below. The file can be found under /etc/scylla/

    • dc - Set the datacenter name
    • rack - Set the rack name For example:
    # cassandra-rackdc.properties
    #
    # The lines may include white spaces at the beginning and the end.
    # The rack and data center names may also include white spaces.
    # All trailing and leading white spaces will be trimmed.
    #
    dc=thedatacentername
    rack=therackname
    # prefer_local=<false | true>
    # dc_suffix=<Data Center name suffix, used by EC2SnitchXXX snitches>
    
  4. After Scylla has been installed and configured, edit scylla.yaml file on all the nodes, using the first node as the seed node. Start the seed node, and once it is in UN state, repeat for all the other nodes, each after the previous is in UN state.

sudo systemctl start scylla-server
  1. Verify that the node has been added to the cluster using nodetool status

Cluster Setup

This example shows how to install and configure a five nodes cluster using GossipingPropertyFileSnitch as the endpoint_snitch, each node on a different rack.

  1. Installing Five Scylla nodes, the IP’s are:
172.21.0.55
172.21.0.56
172.21.0.57
  1. In each Scylla node, edit the scylla.yaml file

    172.21.0.55

    cluster_name: 'msgcloud-prod-ds-sdc'
    seeds: "172.21.0.55"
    endpoint_snitch: GossipingPropertyFileSnitch
    rpc_address: "172.21.0.55"
    listen_address: "172.21.0.55"
    

    172.21.0.56

    cluster_name: 'msgcloud-prod-ds-sdc'
    seeds: "172.21.0.55"
    endpoint_snitch: GossipingPropertyFileSnitch
    rpc_address: "172.21.0.56"
    listen_address: "172.21.0.56"
    

    172.21.0.57

    cluster_name: 'msgcloud-prod-ds-sdc'
    seeds: "172.21.0.55"
    endpoint_snitch: GossipingPropertyFileSnitch
    rpc_address: "172.21.0.57"
    listen_address: "172.21.0.57"
    
  2. This step needs to be done only if using GossipingPropertyFileSnitch. In each Scylla node, edit the cassandra-rackdc.properties file

    172.21.0.55

    # cassandra-rackdc.properties
    #
    # The lines may include white spaces at the beginning and the end.
    # The rack and data center names may also include white spaces.
    # All trailing and leading white spaces will be trimmed.
    #
    dc=msgcloud-prod-ds-sdc
    rack=msgcloud-prod-node01-rack01
    # prefer_local=<false | true>
    # dc_suffix=<Data Center name suffix, used by EC2SnitchXXX snitches>
    

    172.21.0.56

    # cassandra-rackdc.properties
    #
    # The lines may include white spaces at the beginning and the end.
    # The rack and data center names may also include white spaces.
    # All trailing and leading white spaces will be trimmed.
    #
    dc=msgcloud-prod-ds-sdc
    rack=msgcloud-prod-node02-rack01
    # prefer_local=<false | true>
    # dc_suffix=<Data Center name suffix, used by EC2SnitchXXX snitches>
    

    172.21.0.57

    # cassandra-rackdc.properties
    #
    # The lines may include white spaces at the beginning and the end.
    # The rack and data center names may also include white spaces.
    # All trailing and leading white spaces will be trimmed.
    #
    dc=msgcloud-prod-ds-sdc
    rack=msgcloud-prod-node03-rack01
    # prefer_local=<false | true>
    # dc_suffix=<Data Center name suffix, used by EC2SnitchXXX snitches>
    
  3. Starting Scylla nodes, since our seed node is 172.21.0.55 we will start it first, wait until it is in a UN state, and repeat for the other nodes

    sudo systemctl start scylla-server
    
  4. Verify that the node has been added to the cluster by using the nodetool status command

    Datacenter: msgcloud-prod-ds-sdc
    ================================
    Status=Up/Down
    |/ State=Normal/Leaving/Joining/Moving
    --  Address      Load       Tokens       Owns    Host ID                               Rack
    UN  172.21.0.55  1.08 MB    256          ?       9b6435ac-a91b-4c11-bd5c-1e7ef2f6817b  msgcloud-prod-node01-rack01
    UN  172.21.0.57  753.99 KB  256          ?       f1334314-6a9c-43d0-b58e-445bef76918b  msgcloud-prod-node03-rack01
    UN  172.21.0.56  900.26 KB  256          ?       60b36f16-d8a3-457d-8b7c-3c706234de1e  msgcloud-prod-node02-rack01
    
    Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless