Host Administration

Last updated: July 05. 2018


1. Introduction

When setting up the monitoring, certainly the most important task is the administration of the systems to be monitored – the Hosts. It's not just about registering the correct master data (e.g., host name, IP address) – settings for the monitoring (e.g., alarms, threshholds, etc.) also need to be attended to.

Check_MK has been developed from its beginning for environments with a large number of hosts. In order for the configuration to be manageable for the user, Check_MK pursues a different approach to configuration than other systems that originated from the Nagios ecosystem. The most important principles are:

  • a folder hierarchy in which the hosts are stored
  • host characteristics (Host-Tags), and based on these, a rule based configuration
  • Automatic detection of the services to be monitored

1.1. Folders and their hierarchies

Everyone who works with computers knows the principles of data sets and folders. WATO uses a similar principle for administering hosts, which in effect take on the role of data sets. Insofar as folders themselves can be in folders, the result is a 'tree structure'. There are three widely-used criteria for building the host-tree:

  • Location (e.g. Munich versus Shanghai)
  • Host type (e.g. Switch versus Loadbalancer)
  • Organisation structure (e.g. Database group versus Networker)

Naturally you can also mix these criteria in a tree with, for example, subdivision by location in the first level, and by host type in the second.

If you love simple things you should pack the actual hosts only in the tree's "leaves" (although Check_MK also allows hosts in intermediate folders). The following example shows a simple tree structured by host type: The hosts A, B and C are in the folder Servers and D, E and F in Network :

1.2. Attribute inheritance

If you build the tree cleverly you can use it to pass on attributes in a meaningful manner. This is especially useful with attributes that are the same for large groups of hosts, e.g., the SNMP community, or Host characteristics such as Agent type, with which you define whether the host should be monitored per SNMP or per a Check_MK agent.

The following example shows the passing-on of the Agent Type attribute with the cmk-agent and snmp-only values, likewise the Criticality attribute with the prod and test attributes:

Attributes defined lower in the tree always have precedence. Values defined directly at the host therefore overwrite everything that comes from the folders. In the above example, the host A receives the prod and cmk-agent attributes, host D receives prod and snmp-only and host F, because of the explicit attribute test at the host, receives the test and snmp-only values.

A big advantage of this procedure over the the widely-used copy & paste approach of data base oriented configuration systems is that you can PREdefine attributes for hosts that will be registered in the future. This makes your (or your colleagues') work easier – simply throw the new host into the correct folder and all settings will be automatically correct!

1.3. Permissions

A further purpose of the folder is the assignment of permissions for creating and editing hosts. Check_MK here differentiates between rights in WATO and the contact allocation in monitoring. It's not always the case that the persons authorised to create a host are the same people who are responsible for the host's operational monitoring. The permissions are explained in their own article.

You create new folders via the button. The options are the same as when creating new hosts, that we will explain detailedly hereinafter.

2. Creating hosts in WATO

You can manage folders and hosts via the Hosts WATO module:

In the bar-graphic shown below you can always see which folder you are located in:

The create host button, the clone button and edit host button take you to the page with the host's attributes. This consists of three sections:

2.1. The host name

Most important is the host name. Everywhere in Check_MK this field serves to explicitely identify the host. The host name is entered in internal references, used as a component of the URL, serves as a part of file names and indexes, appears in log files, etc. There is in fact a function for changing host names at a later date – this is however a time-consuming and complex procedure that is best avoided. You should therefore select host names carefully. The host's name does not necessarily need to match the host's DNS name, but it makes many things easier.

2.2. Basic settings: alias and IP addresses

In the Basic settings under Alias you can give the host an alternative, descriptive name which will be displayed in many locations in the GUI as well as in reports. If no alias is defined, the host's name will be used as an alias.

You have four options for configuring the IP address:

Option Procedure DNS Action
1 You enter no IP address. The host name must be resolvable via DNS. with Activate changes
2 You enter an IP address – in the standard format. never
3 Instead of an IP address you can alternatively enter a DNS-resolvable host name. during check execution
4 Via rules set Hosts with dynamic DNS lookup during monitoring you determine hosts for a dynamic DNS. The result is similar to 3, except that the host name field is used for DNS query. during check execution

With the host name method Check_MK uses cached data in order to minimise repeated DNS requests during an Activate Changes – which is very important for accelerating the activation procedure. Furthermore, the cache ensures that a changed configuration can still be activated if the DNS stops working.

The catch is that Check_MK doesn't automatically notice the change to an address in DNS. For this reason, in the host details there is the button which deletes the entire DNS cache and forces a new resolution at the next Activate changes. This file is found under ~/var/check_mk/ipaddresses.cache in your instance, by the way. Deleting this file has the same effect as the button as described above.

Check_MK incidentally also supports monitoring via IPv6 – also in Dualstack. Details can be found in its own article.

2.3. Host tags: Check_MK agent or SNMP

The final important setting can be realised in the Host tags box. The attributes shown here can be extended as desired and can be used via rules to configure all host and service parameters very efficiently.

Check_MK automatically creates four groups of attributes, of which Agent type and IP address family are important because these have already been evaluated via existing rules, and are in effect 'armed'. Criticality and Networking segment are examples.

For Agent type the three most important settings are:

Check_MK agentThe host should be monitored via the Check_MK agents (which must be installed of course). Select this setting also in the case of special agents, such as e.g., ESX-Monitoring
SNMPThe host should be monitored via SNMP. This selection allows the SNMP Community field to appear in Basic settings, with which you can define the SNMP-Community. Since this is generally the same for many hosts, it is rather recommended that it be defined in a folder. If nothing is specified 'public' is automatically assumed.
No agent Such hosts are without agents and are monitored only with Active checks. Rules for these are found under Host & Service Parameters ➳ Active checks in WATO. If you don't define at least one active check then Check_MK creates a PING service automatically.

The No agent setting is also the correct one if the host is to be monitored per piggyback technique from another host. This also applies to e.g., VMs from ESX, on which no Check_MK agent ist installed.

2.4. Saving and more

After creating or cloning a host the next logical step is always {{Save & go to Services}}. With this you enter the automatic service detection, a subject we want to address in the next section. Save & Test takes you into the diagnosis mode – with which you can test whether the settings being used produce ANY data at all from the agent. Details about the diagnosis mode can be found in the article on the agents.

3. Configuring services

After creating a host the next step is the configuration of its services to be monitored. All details for the automatic detection and configuration of the services can be found in its own article. We will describe only the most important here.

There are various ways of accessing the list of a host's configured services in WATO:

  • with the Save & go to Services button on a host's detail page
  • with the button on a host's detail page (without saving)
  • with the symbol on the list of hosts in a folder
  • in the menu, selecting the Check_MK Discovery service with the Edit Services entry

A few relevent tips:

  • The usual method when creating a new host is to use the Save manual check configuration button, which adopts all services to be found for monitoring (Available (missing) services).
  • If you open an existing host's page and find services that are not currently being monitored, then the Activate missing button is a sensible tool. This adds the missing services.
  • The Full scan button enables fresh, complete data to be obtained from a target device. Check_MK works with cached data to enable the rapid loading of pages for a normal monitoring's displays. With SNMP devices the button starts an active search for new check plug-ins and can possibly find further services.
  • Automatic Refresh is the same as a clearing and fresh detection of all services. This is useful for services which can recall the state detected with a discovery. (e.g., the current state of switch ports)
  • Via the check boxes you can select or deselect individual services. This is only a temporary solution as the service detection always highlights missing services. To permanently ignore a service requires the creation of a rule and is achieved with the symbol.
  • As always after every change an Activate Changes is necessary in order for them to take effect.
  • All further information can be found in the article on Service configuration.

4. Bulk operations

You may occasionally wish to perform tasks such as deleting, moving, editing or service detection for a whole series of hosts simultaneously. WATO offers so-called bulk operations for this purpose. These always apply for hosts that are located directly in a folder. You can restrict the selection by entering a search text to the left of Search, or via check boxes which you activate with . With a final click on one of the buttons in the Bulk bar the operation will be carried out or at least be initiated for all hosts.

Here are a few tips for the less self-evident operations:

4.1. Edit and cleanup

Edit enables changes to one or more attributes on all selected hosts. The attribute is thereby entered explicitely in the hosts. Attention: there is a difference between the host inheriting an attribute from a folder, and the attribute being set explicitely. Why? In the latter case a change to the attribute in the folder would have no effect, as the values defined directly in the host always have priority.

The Cleanup operation is available for this reason. With this you can delete explicite attributes from the selected hosts and reinstate inheritance. The same result can be achieved by opening every host individually and deselecting the attributes via the check boxes.

It is generally a good idea to use as few explicite attributes as possible. When everything is inherited correctly via the folders, errors are reduced and the easy integration of new hosts is made possible.

4.2. Discovery

You can find details about Discovery in the article on Services.

5. Host searches in WATO

WATO offers its own search function for configured hosts, with which you can search beyond the limits of folders. Why can't you simply search via the views in monitoring? That would certainly work with the search for a single host. You could access this host via the symbol in WATO.

But let us remind ourselves: in the Introduction to WATO article we saw that the hosts in the configuration environment are not necessarily the same as those in the operational monitoring environment. The WATO search additionally offers the possibility of performing bulk operations immediately on the located hosts.

The search can be reached via the button you can find in every folder. The search always preceeds from the current folder recursively through all subfolders. To search globally, simply use the search from the main folder. In the Hostname field an infix search is valid – the entered text must only be a part of the host name. Furthermore, you can restrict the search with characteristics or other attributes:

All search terms are connected with AND. The example in the above image illustrates a search for all hosts with the Test system attribute, that also include ora in their name.

The resulting list behaves almost like a normal folder. This means that here you can work with Bulk operations, in order, for example, to move all hosts found into a specific folder. If you don't like the results, you can adjust and refine the seach at any time with .

6. Importing hosts from CSV data

If you wish to import a large number of hosts from a previous monitoring system or from an Excel table, you can make the task easier by importing with the help of CSV data. Check_MK is very flexible when reading such CSV data. In the simplest case you just need a file in every line of which there is a host name that can be resolved via DNS:

import.csv
myserver01
myserver02
myserver03

During an import it is also possible to take on additional attributes. If the CSV data has attribute names in the first line, Check_MK can even assign these automatically. To this end Check_MK attempts to use a tolerant rather than an exact syntax. In the following data WATO can automatically correlate all four columns correctly:

import.csv
hostname;ip address;alias;agent
srvlnx17;10.0.0.10;web99;cmk-agent
srvlnx18;10.0.0.32;Backupserver;cmk-agent
switch47-11;;Backpserver23;snmp-only

The procedure is as follows: select or create a target folder for the import. Switch to this folder and click on . In the dialogue that opens either upload the data, or select {{Content of CSV file}} and copy the content into the entry field that opens. You can even automatically perform an immediate service discovery on the newly-imported hosts with the Perform automatic service discovery option:

Selecting a separator in the next step is not necessary here, as it will be recognised automatically. Here you select the Has title line option:

A click on Update preview displays the following table:

If the automatic recognition of a column doesn't work you can manually select the attribute to be assigned. Under the host attributes in the CSV data it is essential that the attribute's internal name be used (here e.g. cmk-agent and not Check_MK agent (server)). The exact internal names can found with Host Attributes in the WATO module.

If you have earlier selected Perform automatic service discovery, the same mask as used in Bulk discovery appears. After the discovery comepletes , the only thing missing is the familiar Activate Changes for all of the new hosts to be in monitoring!

7. Creating parents

7.1. Creating parents manually

You have already learned how Parents functions, and what the states of Hosts and Notifications are all about. But how does one actually create Parents? The answer is typically Check_MK: there are a number of different procedures – manually, per scan or via the Web-API.

A parent for a single host can be specified as follows: In WATO ➳ Hosts open the desired host's attributes. In the Basic Settings section enter the parent using its name or IP-address. Once a parent has been specified, a further entry field for an additional parent will be opened.

Important: Only direct Parent-Hosts may be specified.

Similarly, parents can also be defined in a folder's attributes, and be inherited by the hosts they contain. How this is achieved has already been seen in the section on Host-Management.

7.2. Creating parents using a scan

If the monitoring is a new installion, which from the very beginning has been planned with an orderly folder and parents structure, there should be no real problems with the inheriting of parents via folders. Parents can also be set up automatically using a scan. The Parent Scan can be found in WATO ➳ Hosts in each individual folder.

Via the IP-Protocol the scan searches for the last Gateway before a host on the OSI-Model's (Layer 3) Network Layer using traceroute. If such a Gateway is found and its address belongs to one of your monitored hosts, this host will then be set as a parent. If the Hop's traceroute receives no information from the targeted host, the info from the last successful Hop will be used.

If however no gateway with a monitored IP-address is found, as standard Check_MK generates an artificial Ping-only-Host in the Parent folder which will be simultaneously generated by default.

This standard setting can also produce undesirable results. For example, take a typical, small network with the address range 192.168.178.0/24. If a host with an address in a different address space – which cannot be pinged – is added to this monitoring, the scan will attempt to access it via the router, and there it will find only a net-provider node. Thus, for example, it can happen that a telecom-server in the WAN-network is defined as a parent for this host. This option can of course be deactivated.

If you wish to scan a folder with new hosts for parents, proceed as follows:

First navigate to the desired folder and click on the Parent scan icon.

The Scan-Configuration will open. To fully scan in all hosts in all subfolders, independently of possible manually-installed parents, under Selection choose the Include all subfolders and Scan all hosts options. In the Performance menu you can limit the scan-duration – which otherwise can take a very long time if there is a large number of hosts.

In Creation of gateway hosts specify if, how, and under which alias new parent-hosts should be created. Deactivate this function if it is to be restricted to parents on monitored hosts.

Now start the scan. The scan's output can be followed live. On completion the changes must as usual be activated with . Finally the configured parents and, if applicable, a new folder Parents can be viewed in WATO ➳ Hosts.

With this the scan has been completed.

Following a scan process the Parent-Child relationship will be automatically opened as a topological diagram, which can also be displayed with Views ➳ Network Topology.

Tip: If the result of a scan appears to be implausible at any point, a manual traceroute can sometimes help with analysing the individual hops.

By the way – one can also scan selected hosts, rather than a complete folder: in activate the check boxes, select the desired hosts, and start the group action Parentscan.

7.3. Creating parents without WATO

For more experienced users there is the additional facility for configuring parents by using Web-API.

8. Renaming hosts

Renaming hosts – on the face of it a simple matter – turns out to be an astoundingly complex operation on closer inspection. The reason for this is that Check_MK uses the host's name as the unique key for the host – and this is used in numerous locations. These include log data, file names, configuration rules, BI agreggations, reports, dashboards and much more. The host name also appears in URLs.

In order to be able to cleanly rename a host in all locations, WATO has a particular function. In a host's details you can rename it by using the button, or in a folder rename multiple hosts simultaneously with the button.

By utilising intelligent operations, Bulk Renaming allows systematic name matching to be made. In the Hostname matching field you optionally enter a regular expression that matches the first characters of the names of hosts that you wish to rename – here as an example, all hosts whose names begin with mysrv. Then enter one or more operations in the sequence that they should be applied to the hosts. In the following example, for all hosts everything after the first '.' will be truncated and replaced by the ending '.servers':

Numerous operations are available. Please activate the Online Help , and select the operation to receive an explanation about it. Following the obligatory "Are you sure...?" query...

... the processing can take a while. During the renaming the monitoring will be completely stopped! This is necessary to keep everything in a consistent state. On completion you will receive on overview listing which and where renames have taken place:

9. The folder structure in the monitoring view

The tree structure derived from the folders is also visible to their users in monitoring. On the one hand, there is a WATO Folder filter in all views that you can use to restrict the current view to only those hosts below a particular folder:

On the other hand, via the Folders sidebar element you can restrict the view on the right side to a single folder:

This element functions in conjunction with the Views element. Once selected, a folder is retained even if you select another view. This works for dashboards as well. Try it for yourself!