8. General Settings » Graphs & Storage

In a later chapter, you will see how to configure Sensors to generate traffic graphs, top statistics, and accounting information for every IP in the monitored network. In Configuration » General Settings » Graphs & Storage, you can configure how much disk space will be used to store that data, as well as other storage-related settings.

GRAPHS_STORAGE

The panel is divided into three sections: Graphing Engine, which selects the storage backends and the way graphs are rendered; Decoders for IP Graphs, Tops and Accounting, which selects the traffic dissectors whose data is stored; and Advanced Settings, a fieldset that is collapsed by default and holds the storage paths and a few tuning parameters.

The Graph Storage Engine parameters let you select the third-party software used to store graph data:

► InfluxDB is a specialized database designed for time series data. It uses more RAM than RRDTool but requires less storage space, is faster in most cases, can be configured for High Availability, and does not delete existing data when you change the configuration

► ClickHouse is a fast, column-oriented analytics database. It offers the query speed and low storage requirements of InfluxDB while avoiding its cardinality-related scaling issues, so RAM usage stays predictable even when graphing a very large number of IPs or ASNs

► RRDTool is a very stable and old solution for time series data. However, it can be slow in some cases (due to disk seek times), uses more storage space (due to pre-allocation of data), and some configuration changes (e.g., adding a decoder) reset existing graph data

Graph Storage Engine 1 is the primary backend. It can be set to InfluxDB, ClickHouse, RRDTool (On Disk), or RRDTool (In RAM), and by default it is the only backend in use.

Graph Storage Engine 2 is optional. It defaults to None and can be set only to InfluxDB or ClickHouse — RRDTool cannot be used as a secondary backend. When enabled, it stores the same graph data in a second backend simultaneously, for redundancy or for migrating between backends without losing data.

Click the Options button on the right-hand side to configure the selected Graph Storage Engine:

InfluxDB 1.9 is the fastest graph storage engine, but its RAM usage grows with very big series cardinality (>1M).

INFLUXDB_CONFIG

● IP Graph Granularity sets the level of detail for IP, AS, and Country graphs, expressed as a number followed by second(s) or minute(s). The default value is 5 minutes. When using Flow Sensor, avoid setting it lower than the flow delay. This parameter appears only when InfluxDB is Graph Storage Engine 1, because the granularity is dictated by the primary backend
● Storage Restrictions excludes certain classes of graphs from using the selected Graph Storage Engine. This is a multi-select listing every class of graph — Anomaly, Upstream AS, Transit AS, Peering AS, Downstream AS, Country, Filter, Filtering Rules, Profile, IP, Protocols, Sensor, and Server graphs. Select None to apply no restrictions
● InfluxDB URL should be the full URL of the InfluxDB (HTTP/HTTPS) server, for example: http://127.0.0.1:8086
● InfluxDB Username and InfluxDB Password are optional but must be set if auth-enabled=true in influxdb.conf
● InfluxDB Database is also optional. The default database name is andrisoft. Change it only when using a single InfluxDB instance for multiple Consoles
● InfluxDB Connector provides the [Initialize DB], [Delete DB], and [Check Status] buttons. The most important status value is the “cardinality estimation,” which is directly related to InfluxDB’s RAM usage. High-cardinality data includes IP and AS graphs. If RAM usage is too high, consider disabling IP graphing for large subnets, then deleting and reinitializing the database — or switch to ClickHouse, which does not have this limitation

ClickHouse is not constrained by series cardinality, which makes it the most scalable option for IP, AS, and Country graphs.

CLICKHOUSE_CONFIG

● IP Graph Granularity sets the level of detail for IP, AS, and Country graphs, expressed as a number followed by second(s) or minute(s). The default value is 5 minutes, and the maximum is 10 minutes. When using Flow Sensor, avoid setting it lower than the flow delay. This parameter appears only when ClickHouse is Graph Storage Engine 1, because the granularity is dictated by the primary backend
● Storage Restrictions excludes certain classes of graphs from using the selected Graph Storage Engine. This is a multi-select listing every class of graph — Anomaly, Upstream AS, Transit AS, Peering AS, Downstream AS, Country, Filter, Filtering Rules, Profile, IP, Protocols, Sensor, and Server graphs. Select None to apply no restrictions
● ClickHouse URL should be the full URL of the ClickHouse HTTP(S) interface, for example: http://localhost:8123
● ClickHouse Username and ClickHouse Password are required only when authentication is enabled on the ClickHouse server. The default username is default
● ClickHouse Database is optional. The default database name is andrisoft. Change it only when using a single ClickHouse instance for multiple Consoles
● ClickHouse Connector provides the [Initialize DB], [Delete DB], and [Check Status] buttons

ClickHouse Installation

ClickHouse can run on the Console server or on a separate server reachable over HTTP(S). Install the production ClickHouse server from the official repository (for other platforms and installation methods, see the ClickHouse installation documentation):

[root@console ~]# apt-get install -y apt-transport-https ca-certificates curl gnupg
[root@console ~]# curl -fsSL 'https://packages.clickhouse.com/rpm/lts/repodata/repomd.xml.key' | gpg --dearmor -o /usr/share/keyrings/clickhouse-keyring.gpg
[root@console ~]# echo "deb [signed-by=/usr/share/keyrings/clickhouse-keyring.gpg arch=$(dpkg --print-architecture)] https://packages.clickhouse.com/deb stable main" | tee /etc/apt/sources.list.d/clickhouse.list
[root@console ~]# apt-get update
[root@console ~]# apt-get install -y clickhouse-server clickhouse-client
[root@console ~]# systemctl enable --now clickhouse-server

During installation, the clickhouse-server package asks for an optional password for the built-in default user; leaving it empty keeps the user password-less.

[root@console ~]# yum install -y yum-utils
[root@console ~]# yum-config-manager --add-repo https://packages.clickhouse.com/rpm/clickhouse.repo
[root@console ~]# yum install -y clickhouse-server clickhouse-client
[root@console ~]# systemctl enable --now clickhouse-server

Instead of using the built-in default user, you can create a dedicated user for the Console by adding an XML file to the /etc/clickhouse-server/users.d/ directory. First, generate a password together with its SHA-256 hash:

[root@console ~]# PASSWORD=$(base64 < /dev/urandom | head -c12); echo "$PASSWORD"; echo -n "$PASSWORD" | sha256sum | tr -d '-'

Then create /etc/clickhouse-server/users.d/andrisoft.xml with the username as the element name and the hash inside it — a plaintext <password> element also works but is not recommended. All available options are documented in /etc/clickhouse-server/users.xml:

<clickhouse>
    <users>
        <andrisoft>
            <password_sha256_hex>paste the SHA-256 hash here</password_sha256_hex>
            <networks>
                <ip>127.0.0.1</ip>
                <ip>::1</ip>
            </networks>
            <profile>default</profile>
            <quota>default</quota>
        </andrisoft>
    </users>
</clickhouse>

The <networks> element restricts where the user can connect from. Keep it as above when ClickHouse runs on the Console server, or list the IP addresses of the connecting servers otherwise. ClickHouse applies the files in users.d/ automatically, without a restart. Enter the new username and password in the fields described above, then press [Initialize DB].

This method optimizes the long-term storage of IP graph data by allowing up to three Round Robin Archives.

RRDTOOL_DISK

● RRDCached Socket increases the I/O performance of the Console server; see RRDCached Configuration Instructions
● Graph Data Units lets you choose the data units stored inside IP graph files. Any combination of Inbound Packets, Inbound Bits, Outbound Packets, and Outbound Bits can be selected
● Graph Consolidation lets you choose how to build consolidated values for Round Robin Archives. Any combination of Minimum, Average, and Maximum can be selected. If you are interested in traffic spikes, check MAXIMUM. If you are interested in average values, check AVERAGE. For low traffic values, check MINIMUM
● Storage Restrictions parameter excludes several classes of graphs from using the selected Graph Storage Engine. This is a multi-select listing every class of graph — Anomaly, Upstream AS, Transit AS, Peering AS, Downstream AS, Country, Filter, Filtering Rules, Profile, IP, Protocols, Sensor, and Server graphs. Select None to apply no restrictions
● Round Robin Archives determine the granularity of the graphs and the interval of time they are saved. Up to three archives can be defined, in the Accuracy of Round Robin Archives grid, with the [Add Archive], [Modify Archive], and [Delete Archive] buttons; at least one archive must always remain. These entries specify for how long and how accurately the data should be stored. A smaller data average (5 seconds minimum) generates a very accurate graph but requires more disk space, while a bigger data average (5 minutes maximum) is less accurate but uses less disk space. RRDTool stores each graph in a separate file; therefore, on non-SSD drives, the disk seek time may be too high to update thousands of IP graph files every few minutes. If this is the case, configure RRDCached to increase the I/O performance of the Console server; see RRDCached Configuration Instructions. If the speed of updating IP graph files is still not fast enough, consider using another Graph Storage Engine

Graphing Engine Backend lets you choose from which Graph Storage Engine to retrieve data when rendering graphs — either Graph Storage Engine 1 or Graph Storage Engine 2. The Options button lets you select the Graph Rendering method:

► Native (Javascript) renders graphs and charts directly in the browser, enabling click-to-zoom, animations, and rich tooltips

► PNG (High DPI), PNG (Low DPI), and SVG render static images on the Console server. On macOS or Linux, you may need to install the Consolas font to properly display SVG files

Decoders are internal functions (traffic dissectors) that identify and classify the underlying protocols of each packet or flow. Each enabled decoder increases the size of IP graphs, tops, and accounting data, and introduces a small performance penalty, so enable only the decoders you actually need. You can use the existing built-in decoders or you can create custom decoders under General Settings » Custom Decoders.

The Decoders for IP Graphs, Tops and Accounting grid has four columns:

● Enabled – Select the checkbox to store the decoder’s data in IP graphs, tops, and accounting data

● Decoder – The name of the traffic dissector

● Description – A short explanation of what the decoder matches

● Color – The color used for the decoder in graphs. Click the cell to pick a new color

The remaining parameters are located in the Advanced Settings fieldset, which is collapsed by default.

IP Sweep Graphing allows storing IP graph data for IPv4 and/or IPv6 addresses receiving traffic without returning any. It can be set to Off, IPv4, IPv6, or IPv4 and IPv6. Avoid setting it to Off when monitoring unidirectional links or asymmetric traffic. Enabling it for IPv6 is generally not recommended because the vast number of possible IPv6 addresses can quickly exhaust RAM (when using InfluxDB) or disk space (when using RRDTool or ClickHouse).

Sensor Top N (default: 20, minimum 5, maximum 2048) specifies the maximum number of items stored for each ordered data set, such as top talkers, external IPs, ASNs, countries, TCP/UDP ports, IP protocols, etc.

Flow Sensor saves flow data on the local disk in the path defined by the Flow Collector Path, using a directory structure determined by Flow Data Hierarchy:

► Off – No directory hierarchy is created

► Year/Month/Day and Year/Month/Day/Hour – Nested directories, optionally down to the hour

► Year/Week/DoW and Year/Week/DoW/Hour – Nested directories by week and day of the week

► Year/DoY and Year/DoY/Hour – Nested directories by day of the year

► Year-Month-Day and Year-Month-Day/Hour – A single directory per day (for example, 2026-07-09) instead of nested Year, Month, and Day directories

Packet Sensor saves packet dumps in the path set by the Packet Dumps Path.

RRDTool stores all graph files on the Console server in the Graph Disk Path. InfluxDB stores graph data in /var/lib/influxdb/data/, and ClickHouse in /var/lib/clickhouse/. If you want to use another path or drive, symlink it to the original location.

Note

It is strongly recommended to automate the deletion of old data and to monitor disk usage for IP graphs in General Settings » Data Retention.

8.1. Sensor and Applications Graph Troubleshooting

✔ Ensure that all Sensors are running correctly by checking the event log and live statistics in Reports » Devices » Overview. Refer to the troubleshooting guides for Packet Sensor, Flow Sensor, and SNMP Sensor

✔ Discontinuous Sensor graphs can happen if IP Accounting is enabled for too many or too large subnets, especially when there’s a slow connection between the Sensor and the MySQL/MariaDB instance on the Console server

8.2. IP/Subnet and Profiling Graph Troubleshooting

✔ Ensure that all Sensors are running correctly by checking the event log and live statistics in Reports » Devices » Overview Refer to the troubleshooting guides for Packet Sensor, Flow Sensor, and SNMP Sensor
✔ Generating IP and profiling graph data causes the biggest impact on the load of the Console server. Enable each feature (IP graphing, IP accounting, IP profiling) sequentially for each subnet, ensuring the Console server can handle it. The storage requirements for each subnet are listed in the IP Zone, and current disk usage is in General Settings » Data Retention
✔ The internal process used for saving IP graph data is /opt/andrisoft/bin/genrrds_ip. If it is overloading the Console server or the event log contains warnings such as “Updating IP graph data takes longer than 5 minutes”, use ClickHouse, InfluxDB, or RRDCacheD, RAM/SSD updating method, faster disk drivers, enable IP graphing for fewer subnets, or deploy a Sensor Cluster configured to aggregate IP graph data
✔ The internal process used for generating IP or subnet graphs is /opt/andrisoft/bin/gengraph_ip. Console users launch the process for each requested IP or subnet graph. If the Console server gets too loaded by gengraph_ip, execute “killall gengraph_ip” and configure ClickHouse, InfluxDB, or RRDCacheD. When launched, the process stops only when the graph is generated. This process can be slow when users request subnet graphs for subnets not specifically defined in the IP Zone. It is not possible to throttle the number of graphs requested by users

8.3. AS and Country Graph Troubleshooting

✔ Ensure that all Sensors are running correctly by checking the event log and live statistics in Reports » Devices » Overview Refer to the troubleshooting guides for Packet Sensor, Flow Sensor, and SNMP Sensor
✔ To enable AS and Country graphs, set the Stats Engine parameter to either Extended in Flow Sensor’s configuration, or to Full in Packet Sensor’s configuration
✔ SNMP Sensor cannot generate AS or Country graphs due to SNMP protocol constraints