I’ve got a series of nodes which have 3 SNMP-based checks configured on them. I wanted 2 of these checks to depend on 1, so that we only get a single alert if snmpd goes down or if the node drops offline. To do this, I created a servicedependency like so:
define servicedependency{
host_name nodeX
service_description disk_usage
dependent_service_description load_avg_5m, swap_usage
execution_failure_criteria n
notification_failure_criteria u
}As you can see, load_avg_5m and swap_usage depend on disk_usage.
With this configuration in place, I noticed that I was sometimes getting a notification for load_avg_5m, swap_usage and then disk_usage, since this was the order that these services were being checked. Unfortunately, I couldn’t find a way to configure the order in which the services were checked, but fortunately did find the following here:
“*One important thing to note is that by default, Nagios will use the most current hard state of the service(s) that is/are being depended upon when it does the dependeny checks. If you want Nagios to use the most current state of the services (regardless of whether its a soft or hard state), enable the soft_state_dependencies option.”
So, in my case, the current HARD state of disk_usage wasn’t UNKNOWN when the other checks failed, and therefore these services failed first and sent notifications. I’ve since set soft_state_dependencies=1 in /etc/nagios3/nagios.conf, and hope that this helps in reducing the amount of unnecessary notifications I get when there’s a node or snmpd outage.