dmesg: use dmesg to quickly display recent kernel messages. Any warnings about resource starvation, overfilled kernel tables and the like should be addressed by appropriate kernel tuning.
vmstat: vmstat 3 is a quick and easy way to monitor CPU utilization. On a well-utilized Stingray system, the user (us) and system (sy) CPU times will give a rough indication of the utilization, and the idle (id) time a rough indication of the spare capacity. Stingray workload is shared between user and system time; user time will predominate for a complex configuration or one that uses CPU-intensive operations (e.g. SSL, compression), and system time will predominate for a simple configuration with minimal traffic inspection. The wait time (wa) should always be low.
User, System and Idle time is not a good indication of spare capacity because Stingray uses system resources as eagerly as possible, and it will operate more efficiently the more highly-loaded it is. For example, even if a Stingray system is 25% utilized, it will likely be at less than 25% its total capacity.
/proc: use /proc/<pid> for a quick investigation of the state of Stingray processes – memory usage, number of open file descriptors, process limits
ethtool: use ethtool to query and configure the network interface hardware:
- ethtool eth0 - to determine interface speed and negotiated options;
- ethtool –S eth0 to dump statistics for a network interface. Large numbers of retransmits, errors or collisions may indicate a faulty NIC or poor cabling, or a congested network;
- ethtool –i eth0 to confirm the driver in use;
- ethtool –k eth0 to list the offload features employed by the card.
tcpdump: use tcpdump to capture raw packets from named interfaces. A tcpdump analysis can uncover unexpected problems such as slow closes and inappropriate use of tcp optimizations as well as application level problems.
trace: Stingray includes a wrapper (ZEUSHOME/zxtm/bin/trace) around the standard operating system system trace tools. trace will output all system calls performed by the traced Stingray processes.
