Stochastic Geometry

The most robust solution for sending SMS notifications of server issues detected by Nagios is well-known – plug a GSM modem (or a mobile phone) into your server directly and use that as the delivery mechanism. It’s true out-of-band communications, so if the backhaul fails completely, you still get notified.

However, for someone who has a small server like mine, in a geographically remote location (like Germany, in my case), that may be overkill. Especially as the cost includes both the initial cost of the hardware and the ongoing service charges. And for true out-of-band notification, you need far more admin time than may be appropriate for a small site; you have to have scheduled test of out-of-band notifications lest the hardware fail without detection, and so forth.

In my case, the requirement is far lower-spec; I just want to get a quick ping if the server starts to die so I can log in and try to effect repairs. If it’s an actual hardware failure, or an actual backhaul failure, odds are good that I can’t fix it from my end in a few minutes anyway, and we’re past the “fix it” part of the disaster recovery plan (you do have one, right?), and into the “get another temporary webhost and redirect using DNS” part of the DR plan (you do keep regular backups of your site locally, right?).

So my solution is equally lower-spec. I got an account with clickatell.com (there are several other web SMS gateways, I chose clickatell.com because I’m most familiar with them, not because of any objective evaluation) and enabled the HTTP/S API on the account and then wrote the following additions to the Nagios /etc/nagios3/commands.cfg file (again, this is for Debian, your file location may vary…):

[cc lang=”bash” escaped=”true”]define command{
command_name host_notify_with_sms
command_line wget “http://api.clickatell.com/http/sendmsg?user=INSERT_USERNAME_HERE&password=INSERT_PASSWORD_HERE&api_id=INSERT_API_ID_HERE&to=$CONTACTPAGER$&text=’$NOTIFICATIONTYPE$ Server is $HOSTSTATE$ ($HOSTOUTPUT$) @ $LONGDATETIME$'”
}

define command{
command_name service_notify_with_sms
command_line wget “http://api.clickatell.com/http/sendmsg?user=INSERT_USERNAME_HERE&password=INSERT_PASSWORD_HERE&api_id=INSERT_API_ID_HERE&to=$CONTACTPAGER$&text=’$NOTIFICATIONTYPE$ Server : $SERVICEDESC$ is $SERVICESTATE$ @ $LONGDATETIME$'”
}[/cc]

Then tell Nagios to use these new commands in /etc/nagios3/conf.d/contacts_nagios2.conf :

[cc lang=”ini”]define contact{
…
service_notification_commands   service_notify_with_sms notify-service-by-email
host_notification_commands      host_notify_with_sms notify-host-by-email
…
pager                           INSERT_MOBILE_PHONE_NUMBER_HERE
}
[/cc]

And now Nagios warnings and alerts are dispatched both by email (in a longer format) and by SMS. All very simply done, needing only wget to be installed. It’s easily testable as well – just call the wget line from the command line, like so:

[cc lang=”bash” escaped=”true”]wget “http://api.clickatell.com/http/sendmsg?user=INSERT_USERNAME_HERE&password=INSERT_PASSWORD_HERE&api_id=INSERT_API_ID_HERE&to=INSERT_MOBILE_PHONE_NUMBER_HERE&text=’This is a test message. If this was a real message, you would be panicking right now and desperately looking for a terminal.”[/cc]

Given the pricing structure in clickatell, it’s quite affordable – at the rate of normal usage, I should get a year’s coverage for around €50 or so, with capacity to spare. So it’s easy, cheap and testable. It’s not true out-of-band, but as 80% solutions go, it’s not too shabby.