In a later chapter, you will see how to configure Sensors to generate traffic graphs, top statistics, and accounting information for every IP in the monitored network. In Configuration » General Settings » Graphs & Storage, you can configure how much disk space will be used to store that data, as well as other storage-related settings.
The Graph Storage Engine parameters let you select the third-party software used to store graph data:
► InfluxDB is a specialized database designed for time series data. It uses more RAM than RRDTool but requires less storage space, is faster in most cases, can be configured for High Availability, and does not delete existing data when you change the configuration
► RRDTool is a very stable and old solution for time series data. However, it can be slow in some cases (due to disk seek times), uses more storage space (due to pre-allocation of data), and some configuration changes (e.g., adding a decoder) reset existing graph data
Wanguard uses Graph Storage Engine 1 as the only backend by default. However, it can be configured to use two backends simultaneously in these combinations: RRDTool + InfluxDB, or InfluxDB (primary) + InfluxDB (secondary)
Click the Options button on the right-hand side to configure the selected Graph Storage Engine:
InfluxDB 1.9 is the preferred choice for new installations.
● IP Graph Granularity sets the level of detail for IP, AS, and Country graphs. The default value is 5 minutes. When using Flow Sensor, avoid setting it lower than the flow delay
● Storage Restrictions excludes certain classes of graphs from using the selected Graph Storage Engine
● InfluxDB URL should be the full URL of the InfluxDB (HTTP/HTTPS) server, for example: http://127.0.0.1:8086
● InfluxDB Username and InfluxDB Password are optional but must be set if auth-enabled=true in influxdb.conf
● InfluxDB Database is also optional. The default database name is andrisoft. Change it only when using a single InfluxDB instance for multiple Consoles
● InfluxDB Connector lets you initialize, delete, or check the status of the database. The most important status value is the “cardinality estimation,” which is directly related to InfluxDB’s RAM usage. High-cardinality data includes IP and AS graphs. If RAM usage is too high, consider disabling IP graphing for large subnets, then deleting and reinitializing the database
This method optimizes the long-term storage of IP graph data by allowing up to three Round Robin Archives.
● Graph Data Units lets you choose the data units stored inside IP graph files
● Graph Consolidation lets you choose how to build consolidated values for Round Robin Archives. If you are interested in traffic spikes, check MAXIMUM. If you are interested in average values, check AVERAGE. For low traffic values, check MINIMUM
● Storage Restrictions parameter excludes several classes of graphs from using the selected Graph Storage Engine
● Round Robin Archives determine the granularity of the graphs and the interval of time they are saved. These entries specify for how long and how accurately the data should be stored. A smaller data average (5 seconds minimum) generates a very accurate graph but requires more disk space, while a bigger data average (5 minutes maximum) is less accurate but uses less disk space. RRDTool stores each graph in a separate file; therefore, on non-SSD drives, the disk seek time may be too high to update thousands of IP graph files every few minutes. If this is the case, configure RRDCached to increase the I/O performance of the Console server; see RRDCached Configuration Instructions. If the speed of updating IP graph files is still not fast enough, consider using another Graph Storage Engine
This method is not optimal for long-term storage because it allows a single Round Robin Archive per IP graph file. Select this method when you can’t use InfluxDB and when the previous method (configured with RRDCached) is not fast enough to sustain updating thousands of very high-granularity IP graphs.
● Graphs RAM Path is where the graph files are created, updated, and moved periodically onto a larger, slower disk
● Graph Data Units lets you choose the data units stored inside IP graph files
● Graph Consolidation lets you choose how to build consolidated values for Round Robin Archives. If you are interested in traffic spikes, check MAXIMUM. If you are interested in average values, check AVERAGE. For low traffic values, check MINIMUM
● Storage Restrictions parameter excludes several classes of graphs from using the selected Graph Storage Engine
Graphing Engine Backend lets you choose from which Graph Storage Engine to retrieve data when rendering graphs. The options button lets you select which graph file format to use: SVG or PNG. On macOS or Linux, you may need to install the Consolas font to properly display SVG files.
Decoders are internal functions (traffic dissectors) that identify and classify the underlying protocols of each packet or flow. Each enabled decoder increases the size of IP graphs, tops, and accounting data, and introduces a small performance penalty, so enable only the decoders you actually need. You can use the existing built-in decoders or you can create custom decoders under General Settings » Custom Decoders.
IP Sweep Graphing allows storing IP graph data for IPv4 and/or IPv6 addresses receiving traffic without returning any. Avoid setting it to Off when monitoring unidirectional links or asymmetric traffic. Enabling it for IPv6 is generally not recommended because the vast number of possible IPv6 addresses can quickly exhaust RAM (when using InfluxDB) or disk space (when using RRDTool).
Sensor Top N (default: 20) Specifies the maximum number of items stored for each ordered data set, such as top talkers, external IPs, ASNs, countries, TCP/UDP ports, IP protocols, etc.
Flow Sensor saves flow data on the local disk in the path defined by the Flow Collector Path, using a directory structure determined by Flow Data Hierarchy.
Packet Sensor saves packet dumps in the path set by the Packet Dump Path.
RRDTool stores all graph files on the Console server in the Graphs Disk Path. InfluxDB stores graph data in /var/lib/influxdb/data/. If you want to use another path or drive, symlink it to the original location.
Note
It is strongly recommended to automate the deletion of old data and to monitor disk usage for IP graphs in General Settings » Data Retention..
8.1. Sensor and Applications Graph Troubleshooting¶
✔ Discontinuous Sensor graphs can happen if IP Accounting is enabled for too many or too large subnets, especially when there’s a slow connection between the Sensor and the MySQL/MariaDB instance on the Console server.
8.2. IP/Subnet and Profiling Graph Troubleshooting¶
✔ Generating IP and profiling graph data causes the biggest impact on the load of the Console server. Enable each feature (IP graphing, IP accounting, IP profiling) sequentially for each subnet, ensuring the Console server can handle it. The storage requirements for each subnet are listed in the IP Zone, and current disk usage is in General Settings » Data Retention
✔ The internal process used for saving IP graph data is /opt/andrisoft/bin/genrrds_ip. If it is overloading the Console server or the event log contains warnings such as “Updating IP graph data takes longer than 5 minutes”, use InfluxDB or RRDCacheD, RAM/SSD updating method, faster disk drivers, enable IP graphing for fewer subnets, or deploy a Sensor Cluster configured to aggregate IP graph data
✔ The internal process used for generating IP or subnet graphs is /opt/andrisoft/bin/gengraph_ip. Console users launch the process for each requested IP or subnet graph. If the Console server gets too loaded by gengraph_ip, execute “killall gengraph_ip” and configure InfluxDB or RRDCacheD. When launched, the process stops only when the graph is generated. This process can be slow when users request subnet graphs for subnets not specifically defined in the IP Zone. It is not possible to throttle the number of graphs requested by users
✔ To enable AS and Country graphs, set the Stats Engine parameter to either Extended in Flow Sensor’s configuration, or to Full in Packet Sensor’s configuration
✔ SNMP Sensor cannot generate AS or Country graphs due to SNMP protocol constraints.
