Disk Space Configuration

QoE Stor uses several folders for storage:

• /var/qoestor/backend/dump – temporary data received by receivers is stored here. After being loaded into the database, this data is deleted.
• /var/lib/clickhouse/ - the so-called DEFAULT disk. Here, the database schema and data (raw and aggregated logs) are stored.
• /var/lib/clickhouse-hot/ - the so-called HOT disk.
• /var/lib/clickhouse-cold/ - the so-called COLD disk.

HOT and COLD disks are used when storage system cost reduction is needed.

DEFAULT must be very fast (e.g., NVMe SSD) to receive data from DPI, save it to the database in raw logs, and aggregate it (form aggregated logs).

The HOT disk can store aggregated logs for 1-2 weeks. The HOT disk should also be an SSD to quickly generate reports.

The COLD disk can store old data. COLD can be a SATA HDD disk, as reports on old data are rarely generated, and data can be stored on a slow disk.

For instructions on physically separating directories across disks, see below in the section Configuring Storage on a Separate Disk.

By default, all data is stored in the /var partition.

Assume we have mounted a separate disk to /storage.

1. Work as the root user:

   sudo su

2. Stop the receivers and database:

   fastor-stop
   fastor-db-stop

3. Create directories in the /storage partition:

   mkdir /storage/qoestor
   mkdir /storage/qoestor/clickhouse
   mkdir /storage/qoestor/clickhouse-hot
   mkdir /storage/qoestor/clickhouse-cold
   mkdir /storage/qoestor/dump

4. Copy data to the new disk:

   cp -r /var/lib/clickhouse/* /storage/qoestor/clickhouse
   cp -r /var/lib/clickhouse-hot/* /storage/qoestor/clickhouse-hot
   cp -r /var/lib/clickhouse-cold/* /storage/qoestor/clickhouse-cold
   cp -r /var/qoestor/backend/dump/* /storage/qoestor/dump

5. Change the owner of the /storage/qoestor/clickhouse folder:

   chown -R clickhouse:clickhouse /storage/qoestor/clickhouse
   chown -R clickhouse:clickhouse /storage/qoestor/clickhouse-hot
   chown -R clickhouse:clickhouse /storage/qoestor/clickhouse-cold

6. Remove the old directories:

   rm -rf /var/lib/clickhouse
   rm -rf /var/lib/clickhouse-hot
   rm -rf /var/lib/clickhouse-cold
   rm -rf /var/qoestor/backend/dump

7. Create symlinks:

   ln -s /storage/qoestor/clickhouse /var/lib/clickhouse
   ln -s /storage/qoestor/clickhouse-hot /var/lib/clickhouse-hot
   ln -s /storage/qoestor/clickhouse-cold /var/lib/clickhouse-cold
   ln -s /storage/qoestor/dump /var/qoestor/backend/dump

8. Check the links:

   readlink -f /var/lib/clickhouse
   readlink -f /var/lib/clickhouse-hot
   readlink -f /var/lib/clickhouse-cold
   readlink -f /var/qoestor/backend/dump

9. Start the database:

   fastor-db-restart

10. Start the receivers:

    fastor-restart