We have implemented Grafana along with our Nagios plugin AAG, for quite some time. However, this was done using pnp4Nagios to capture the performance data from our checks and push them through to a PNP data source in Grafana. This worked well with our custom-built stack of Nagios Core (the free open-source version of Nagios). But… issues arose recently when we were asked to implement Grafana into a Nagios XI environment.
To start, we simply installed pnp4nagios on our local NagiosXI system and went to connect Grafana the same way we always have. While working through the installation, it occurred to us that the graphs provided by NagiosXI, would stop working as pnp4nagios would be called to handle performance data instead of letting NagiosXI handle it. Reaching out to the customer, they wanted to continue to use the Nagios provided graphs so this solution did not fit their requirements… Back to the drawing board…
This was when we delved into the APIs that NagiosXI provides! These will not only return the data back in JSON format, but they will also allow you to call the APIs from a remote system should you want to separate your Nagios environment and Grafana.
The documentation for these APIs can be found in the help section of your NagiosXI window.
The list of APIs available included things like Service Status, Host status, State History, etc… We figured calling these APIs may return the data we are looking to graph!
Calling APIs like “servicestatus” returned a pile of very useful data. Unfortunately, we noticed that the data returned would only ever be a snapshot in time, not a history of the data over time… That does not work well for graphs….
Thinking back to how pnp4Nagios handled the data, we remembered that the data is all stored in RRD files with a data frame format. Conveniently, there was an API called “rrdexport”!! Using this API while passing in parameters to narrow the scope of the returned data we can pull back the contents of a RRD file in JSON format.
Now that we have the data, we need to be able to push it through to Grafana to build into graphs. I am sure there are many ways of doing this, however, we found the data source called “Infinity” which turns JSON, returned from a HTTP request, into a usable data source for graphing.
Here is an example of a Dashboard built in Grafana using pnp4Nagios:
And finally here is the same dashboard but built using API calls:
These two Dashboards are almost indistinguishable (minus the fact we are graphing two different systems of course) …. We have also found no noticeable performance hits by using this method! So, I would consider this a Win-Win!
Since we are now hosting a remote Grafana instance, we thought it would be ideal if we could open it up to customers for viewing… This would of course require the Grafana instance to be locked down. Luckily Grafana offers three different user levels:
By categorizing dashboards by “Organization” we can lock users into only seeing dashboards related to their monitoring and we can also allocate admins within their staff which will be able to work with their dashboards but not interact with other customers.
Find more info here: https://grafana.com/docs/grafana/latest/administration/roles-and-permissions/
If you wish to implement this into your Nagios environment I believe this is a good starting point, OR reach out to us and we will be able to help you configure it for yourself.
Cheers!
Charlie