Sharding configuration [Документация VAS Experts]
Документация VAS Experts Документация VAS Experts-mobile
in

Settings

  • Show page
  • Old revisions
  • Export to PDF
  • Fold/unfold all
  • Backlinks
  • Show page
  • Old revisions
  • Export to PDF
  • Fold/unfold all
  • Backlinks

    • Differences

    This shows you the differences between two versions of the page.

    Link to this comparison view

    Both sides previous revisionPrevious revision
    Next revision    		Previous revision
    en:dpi:dpi_components:qoestor:configuration:sharding:start [2023/04/03 13:04] – arusnaken:dpi:dpi_components:qoestor:configuration:sharding:start [Unknown date] (current) – removed - external edit (Unknown date) 127.0.0.1
    Line 1: Line 1:
    -====== Sharding configuration ====== 
    -{{indexmenu_n>7}} 
    - 
    -QoE Stor can work in sharding mode: horizontal cluster scaling, in which parts of the same database are placed on different shards (nodes). 
    -Through sharding, you can: 
    -1. Distribute the load on the IPFIX inserts from several DPIs, spreading it over the nodes. 
    -2. At times speed up the construction of reports by increasing the machine resources-nodes: the construction of reports is performed in parallel and glued into the final report. 
    - 
    -===== 1 Create a cluster ===== 
    - 
    -To create a QoE Stor cluster, you must specify the cluster node settings on each node in the file /etc/clickhouse-server/config.d/qoestor-cluster-config.xml 
    - 
    -Edit file on 1st node 
    - 
    -{{ :dpi:dpi_components:qoestor:configuration:sharding:sharding-xml.png?direct&400 |}} 
    - 
    -For each shard, you must specify the host IP or its name. Save your changes. 
    - 
    -Copy the file to the rest of the cluster nodes. 
    - 
    -Restart the database with the <code>fastor-db-restart</code> command on each node. The cluster is ready to run. 
    - 
    -<note important>Port 9000 must be open on all firewall cluster nodes in the internal network between the cluster nodes.</note> 
    - 
    - 
    -===== 2 Adding a cluster to the GUI ===== 
    - 
    -==== 1 Master node connection ==== 
    - 
    -Choose which node you will use as the master. Enable it in the GUI settings. Enable cluster mode. See screenshot below. 
    - 
    -{{ :dpi:dpi_components:qoestor:configuration:sharding:sharding-master-node.png?direct&400 |}} 
    - 
    -<note important>Port 8123 must be open on all cluster nodes.</note> 
    - 
    -==== 2 Management via SSH ==== 
    - 
    -In the GUI Administrator / Hardware section, add all nodes for SSH control. 
    - 
    -{{ :dpi:dpi_components:qoestor:configuration:sharding:sharding-ssh.png?direct&400 |}} 
    - 
    -==== 3 Setting up IPFIX receivers ==== 
    - 
    -In the QoE Stor configuration, configure the receivers for each node. 
    - 
    -{{ :dpi:dpi_components:qoestor:configuration:sharding:sharding-receivers.png?direct&400 |}} 
    - 
    -Reconfigure DPI to send IPFIX to the appropriate QoE cluster nodes. 
    - 
    -<note important>Distribute the load evenly. One or more DPIs must send IPFIX to one QoE Stor node.</note>   
    - 
    -<note warning>Do not send IPFIX from a single DPI to multiple QoE Stor nodes.</note> 
    -