Data export [Документация VAS Experts]

This is an old revision of the document!


Data export

If necessary, you can independently generate your own reports without additional tools and upload data in any CSV, JSON, TabSeparated format.

Data is stored in 4 main logs

  • qoestor.fullflow – full netflow log, storage period - 2 hours by default
  • qoestor.clicksteam – full clickstream log, storage period - 2 hours by default
  • qoestor.fullflow_agg – pre-aggregated netflow log, storage period - 14 days by default
  • qoestor.clicksteam_agg – pre-aggregated clickstream log, storage period - 14 days by default

The command format is as follows

clickhouse-client --database=qoestor  --query="Your sql is here"

By default data is uploaded in TabSeparated format.

Example. The client asked for a log of connections to a specific host in CSV format

clickhouse-client --database=qoestor --query="select * from fullflow prewhere flow_start_date = '2018-10-04' where (source_ipv4 = '10.64.66.100' or destination_ipv4 = '10.64.66.100') and host = 'google.com' ORDER BY flow_start_time limit 10 format CSV"

For detailed information on SQL ClickHouse, see the link https://clickhouse.com/docs/en/sql-reference/statements/select/