Integration with Nagios
Last updated: July 11. 2012
1. Event Status of a Host as a Nagios service
You can have the current event status of a host be reflected by a service in the (status based) monitoring. That service will be CRIT if at least one open or acknowledged event of the state CRIT exists for a host (UNKNOWN or WARN likewise). It is also possible to split this up into several checks based upon the application field of the event.
This is realized by an active check named check_mkevents. If you're using OMD builds from our subscription, then the plugin is found in your site in lib/nagios/plugins. For a short help simply call it without arguments:
OMD[mysite]:~$ lib/nagios/plugins/check_mkevents Usage: check_mkevents [-H REMOTE:PORT] HOST [APPLICATION]
By specifying a host name or IP address you can query for events of that host:
OMD[mysite]:~$ lib/nagios/plugins/check_mkevents zabc123 WARN - 12 events (8 unacknowledged), worst state is WARN OMD[mysite]:~$ echo $? 1
It is also possible to just query events of a certain application. Here you can use regular expressions:
OMD[mysite]:~$ lib/nagios/plugins/check_mkevents zabc123 foobar OK - 0 events
If you like using WATO then you will find a new rule that allows to conveniently configure this check for your desired hosts. The easiest way to find this rule is with the new search feature:
In most cases it is sufficient to just create one rule. In the case that the events in question do not use the host's monitoring name but an IP address as source host, you can select Host IP Address as host specification.
It is even possible to explicitely specify a host name. In that case you need one rule per host, however. But it helps in cases where the host names stored in the event do not match the host names in your monitoring.
After saving and activating your configuration you will get one (or several) services for each host that matches the rule. This will be checked via active checks on a regular basis for its event status.
2. Notify into the Event Console
The opposite way of integration is to have the monitoring send its notifications into the Event Console. This is useful in setups where the notifications do not go to dedicated persons but to a operations center where a team deals with the incoming events.
This kind of integration is configured in two steps. First you configure the monitoring to send notifications for certain hosts and services into the Event Console. Then you create rules in the Event Console that deal with these alerts and create (and maybe also cancel) events.
As a preparatory work you first need to create or select a contact group and put those hosts and services into that group whose notifications you want to go the the Event Console. This intermediate step is a flexible way to distinguish between critical and less critical systems, so that not every little monitoring issue raises an event. In the following screen shot I've called this contact group Event Console:
Note: please make sure that at least one host or service is member of this contact group. Otherwise the group will not be created and your configuration is invalid!
Now you can make the following settings:
Send notifications to Event Console: Enable notification by selecting a contact group here.
Syslog facility for Event Console notifications: WATO will make the notification bear this syslog facility. Have this in mind when you later create the matching rules. Default is to use local0.
Forward notifications to remote host: This is needed in distributed setups.
As soon as you activate these changes all alerts from hosts and services in the selected contact group will be sent as messages to the Event Console (this is in addition to potential other notifications). These messages will contain the most important informations about the alert:
2.1. Using the Check_MK Micro Core
New in 1.2.3i3: If you are using the Check_MK Micro Core, then the forwarding to the Event Console cannot be done by the described global options. Instead you make use of the notification plugin Forward Notification to Event Console. For that purpose you need to create a dedicated user that is member of the required contact group(s). The select Flexible Notifications and the plugin Forward Notification to Event Console. The syslog facility can be set by the first parameter. Please enter 16 for local0, 17 for local1, ... and 23 for local7. An optional remote host (its IP address) can be entered into the second parameter.
2.2. Create matching rules
The next task is to create rules that collect these messages and create corresponding events. This can be done by one rule with the following settings:
3. Avoiding Loops
If you use both event state monitoring and notifications into the Event Console you might end up in an infinite loop because the notifications from the "Events" service will itself create an event. An easy way to avoid this is creating a second rule with the following settings and put this before the other rule:
Testing the notifications into the Event Console can be done in several ways. One easy way is to use command on services to Fake check results and thus manually set services to CRIT and back to OK. If you do this then please be aware of Nagios' flap detection. After several backs and forths the services switches to flapping. In this state any further state change will send out no notification!