This is an old revision of the document!
Disk Space Configuration
QoE Stor uses several folders for storage:
- /var/qoestor/backend/dump – temporary data received by receivers is stored here. After being loaded into the database, this data is deleted.
- /var/lib/clickhouse/ - the so-called DEFAULT disk. Here, the database schema and data (raw and aggregated logs) are stored.
- /var/lib/clickhouse-hot/ - the so-called HOT disk.
- /var/lib/clickhouse-cold/ - the so-called COLD disk.
HOT and COLD disks are used when storage system cost reduction is needed.
DEFAULT must be very fast (e.g., NVMe SSD) to receive data from DPI, save it to the database in raw logs, and aggregate it (form aggregated logs).
The HOT disk can store aggregated logs for 1-2 weeks. The HOT disk should also be an SSD to quickly generate reports.
The COLD disk can store old data. COLD can be a SATA HDD disk, as reports on old data are rarely generated, and data can be stored on a slow disk.
For instructions on physically separating directories across disks, see below in the section "Configuring Storage on a Separate Disk".
Configuring Storage on a Separate Disk
By default, all data is stored in the /var partition.
Assume we have mounted a separate disk to /storage.
- Work as the root user:
sudo su
- Stop the receivers and database:
fastor-stop fastor-db-stop
- Create directories in the /storage partition:
mkdir /storage/qoestor mkdir /storage/qoestor/clickhouse mkdir /storage/qoestor/clickhouse-hot mkdir /storage/qoestor/clickhouse-cold mkdir /storage/qoestor/dump
- Copy data to the new disk:
cp -r /var/lib/clickhouse/* /storage/qoestor/clickhouse cp -r /var/lib/clickhouse-hot/* /storage/qoestor/clickhouse-hot cp -r /var/lib/clickhouse-cold/* /storage/qoestor/clickhouse-cold cp -r /var/qoestor/backend/dump/* /storage/qoestor/dump
- Change the owner of the /storage/qoestor/clickhouse folder:
chown -R clickhouse:clickhouse /storage/qoestor/clickhouse chown -R clickhouse:clickhouse /storage/qoestor/clickhouse-hot chown -R clickhouse:clickhouse /storage/qoestor/clickhouse-cold
- Remove the old directories:
rm -rf /var/lib/clickhouse rm -rf /var/lib/clickhouse-hot rm -rf /var/lib/clickhouse-cold rm -rf /var/qoestor/backend/dump
- Create symlinks:
ln -s /storage/qoestor/clickhouse /var/lib/clickhouse ln -s /storage/qoestor/clickhouse-hot /var/lib/clickhouse-hot ln -s /storage/qoestor/clickhouse-cold /var/lib/clickhouse-cold ln -s /storage/qoestor/dump /var/qoestor/backend/dump
- Check the links:
readlink -f /var/lib/clickhouse readlink -f /var/lib/clickhouse-hot readlink -f /var/lib/clickhouse-cold readlink -f /var/qoestor/backend/dump
- Start the database:
fastor-db-restart
- Start the receivers:
fastor-restart
Was this information helpful?