TheRosiek.com Random tech notes and tutorials

6Oct/100

Nagios Setup on Debian Lenny

Below are instructions on building Nagios from scratch on Debian Lenny. These instructions also make it very easy to update the build when new ones come out.


Apache Configuration

First install the essential packages needed to compile and use Nagios.

apt-get install apache2 php5 openssl xfsprogs build-essential autoconf libgd2-xpm-dev libssl-dev ntpdate libperl-dev libnet-snmp-perl libdbd-sybase-perl libxml2-dev libmysqlclient15-dev mailx

Modify the file /etc/apache2/apache2.conf so that it's a bit cleaner.

ServerRoot "/etc/apache2"

LockFile /var/lock/apache2/accept.lock
PidFile ${APACHE_PID_FILE}
Timeout 300
KeepAlive On
MaxKeepAliveRequests 25
KeepAliveTimeout 15

<IfModule mpm_prefork_module>
    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
    MaxClients          150
    MaxRequestsPerChild   0
</IfModule>

<IfModule mpm_worker_module>
    StartServers          2
    MaxClients          150
    MinSpareThreads      25
    MaxSpareThreads      75
    ThreadsPerChild      25
    MaxRequestsPerChild   0
</IfModule>

User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}

AccessFileName .htaccess
<Files ~ "^\.ht">
    Order allow,deny
    Deny from all
</Files>

DefaultType text/plain
HostnameLookups Off

ErrorLog /var/log/apache2/error.log
LogLevel warn

# Include module configuration:
Include /etc/apache2/mods-enabled/*.load
Include /etc/apache2/mods-enabled/*.conf

# Include all the user configurations:
Include /etc/apache2/httpd.conf

# Include ports listing
Include /etc/apache2/ports.conf

LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

CustomLog /var/log/apache2/other_vhosts_access.log vhost_combined

# Include generic snippets of statements
Include /etc/apache2/conf.d/

# Include the virtual host configurations:
Include /etc/apache2/sites-enabled/

Now create the Apache configuration file for Nagios so it can load the web interface. Create the file /etc/apache2/conf.d/nagios.conf with the following:

ScriptAlias /nagios/cgi-bin "/usr/local/nagios/sbin"

<Directory "/usr/local/nagios/sbin">
   Options ExecCGI
   AllowOverride None
   Order allow,deny
   Allow from all
   AuthName "Nagios Access"
   AuthType Basic
   AuthUserFile /etc/nagios/htpasswd.users
   Require valid-user
   SetEnv TZ "US/Eastern"
</Directory>

Alias /nagios "/usr/local/nagios/share"

<Directory "/usr/local/nagios/share">
   Options None
   AllowOverride None
   Order allow,deny
   Allow from all
   AuthName "Nagios Access"
   AuthType Basic
   AuthUserFile /etc/nagios/htpasswd.users
   Require valid-user
</Directory>

Modify /etc/apache2/ports.conf to have just one line to listen for SSL:

Listen 443

Modify /etc/apache2/mods-available/ssl.conf to clean up the SSL configuration, we'll enable this later.

<IfModule mod_ssl.c>

SSLRandomSeed startup builtin
SSLRandomSeed startup file:/dev/urandom 512
SSLRandomSeed connect builtin
SSLRandomSeed connect file:/dev/urandom 512

AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl    .crl
SSLPassPhraseDialog  builtin
SSLSessionCache        shmcb:/var/run/apache2/ssl_scache(512000)
SSLSessionCacheTimeout  300
SSLMutex  file:/var/run/apache2/ssl_mutex
SSLCipherSuite HIGH:MEDIUM:!ADH
SSLProtocol all -SSLv2

</IfModule>

Create the file /etc/apache2/sites-available/nagios-ssl for the virtual directory routing information.

NameVirtualHost *
<VirtualHost *>
        ServerAdmin webmaster@localhost

        CustomLog /var/log/apache2/access.log combined

        SSLEngine on
        SSLCertificateFile    /etc/ssl/private/apache-self-signed.crt
        SSLCertificateKeyFile /etc/ssl/private/apache-priv.key

</VirtualHost>

Modify the file /etc/apache2/httpd.conf so it has just one line:

ServerName server.domain.com

Now we can enable our modules and sites in Apache.

a2enmod ssl
a2dissite default
a2ensite nagios-ssl

Now create a self signed certificate to use for the site, or a third party if necessary.

Nagios Install

Prerequisites

Setup our users and groups that will run Nagios. Note that the www-root user is specific to Debian since that user runs the Apache web process:

groupadd -g 9000 nagios
groupadd -g 9001 nagcmd
usermod -G nagcmd www-data
useradd -u 9000 -g nagios -G nagcmd -d /usr/local/nagios -c "Nagios Admin" nagios

Now setup the initial directories and permissions where we'll keep the files. I chose to create /var/nagios because that's where I'll store the data files since it's the largest partition and meant for variable data.

mkdir /usr/local/nagios/ /etc/nagios/ /var/nagios
chown nagios:nagios /usr/local/nagios /etc/nagios /var/nagios

Download the Nagios source and unpack.

cd /usr/src
wget http://superb-east.dl.sourceforge.net/sourceforge/nagios/nagios-3.x.x.tar.gz
tar zxvf nagios-3.x.x.tar.gz
cd nagios-3.x.x

Install

Configure and build Nagios for the proper directories and options.

./configure --sysconfdir=/etc/nagios --localstatedir=/var/nagios --with-command-group=nagcmd --with-perlcache --enable-embedded-perl
make all
make install
make install-init
make install-commandmode
make install-config

Set Nagios to start on boot and set the password Apache file to authenticate users.

update-rc.d nagios defaults 99
cd /etc/nagios/
htpasswd -c htpasswd.users nagios
chown www-data htpasswd.users
chmod 600 htpasswd.users

To add more users to the htpasswd.users file just type

htpasswd htpasswd.users username

since the -c is only for file creation.

Configuration

Edit the file /etc/nagios/cgi.cfg and modify any "authorized" line to include the users you created in the htpassswd.users file, for example:

authorized_for_system_information=nagios,user1,user2

Create a folder in /etc/nagios to contain all the configuration files. Typically the site name is a good choice. Once the folder is created, modify the file /etc/nagios/nagios.cfg and remove all the "cfg_file" entries and add the line:

cfg_dir=/etc/nagios/site1

Or whatever folder you created for your config files, site1 in this example.

Now add some images for icon view to the base install. These packs can usually be downloaded from the Nagios Exchange.

cd /usr/local/nagios/share/images/logos
tar zxvf image_logos.tar.gz

Third Party Tools

Plugins

The Nagios plugins are necessary to run checks on server processes and health. In the configure statement, be sure to use the same folder locations as when Nagios itself was configured. Plugins will be installed in the /usr/local/nagios/libexec directory.

cd /usr/src
wget http://superb-east.dl.sourceforge.net/sourceforge/nagios-plugins/nagios-plugins-1.4.xx.tar.gz
tar zxvf nagios-plugins-1.4.xx.tar.gz
cd nagios-plugins-1.4.xx
./configure --sysconfdir=/etc/nagios --localstatedir=/var/nagios --enable-perl-modules
make
make check
make install

NRPE

This install adds the NRPE plugin so that checks can be done on other servers securely and remotely. The NRPE client/server doesn't need to be installed on the Nagios server; it just needs the plugin to run the checks. The remote servers of course need the NRPE client installed. Again, take note of the configure directors and that --enable-ssl=yes is set to yes. This allows for a secure SSL handshake.

cd /usr/src
wget http://superb-east.dl.sourceforge.net/sourceforge/nrpe/nrpe-2.xx.tar.gz
tar zxvf nrpe-2.xx.tar.gz
cd nrpe-2.xx
./configure --sysconfdir=/etc/nagios --localstatedir=/var/nagios --enable-ssl=yes
make all
make install

PNP Graphing

A couple packages are needed first as well as mod_rewrite for Apache:

apt-get install rrdtool librrds-perl php5-gd
a2enmod rewrite

Open /etc/php5/apache2/php.ini and make sure the following is set to Off:

magic_quotes_gpc = Off

Then restart Apache to have the setting take effect:

/etc/init.d/apache2 restart

Download, configure and install PNP. Note directory locations in configure.

cd /usr/src
wget http://voxel.dl.sourceforge.net/project/pnp4nagios/PNP-0.6/pnp4nagios-0.6.x.tar.gz
tar zxvf pnp4nagios-0.6.x.tar.gz
cd pnp4nagios-0.6.x
./configure --sysconfdir=/etc/pnp --localstatedir=/var/pnp4nagios
make all
make install
make install-webconf
make install-config

Setup the base configuration:

cd /etc/pnp
cp process_perfdata.cfg-sample process_perfdata.cfg
cp rra.cfg-sample rra.cfg
cd /etc/nagios

Edit /etc/nagios/nagios.cfg to turn on some switches and make performance data go to PNP. Typically these are commented out to start.

service_perfdata_command=service-perfdata-pnp
process_performance_data=1

Add the new command to commands.cfg in /etc/nagios/site1 or wherever your commands for the site are stored:

define command {
     command_name    service-perfdata-pnp
     command_line    /usr/bin/perl /usr/local/pnp4nagios/libexec/process_perfdata.pl
}

Modify /etc/apache2/conf.d/pnp4nagios.conf to include the same password file that Nagios uses. This file is created by the install scripts.

AuthUserFile /etc/nagios/htpasswd.users

Restart Apache for the changes to take effect. You can now navigate to https://hostname/pnp4nagios and see the install checklist. If all is complete, remove the /usr/local/pnp4nagios/share/install.php file.

General Use

For Nagios to function, it first needs to be started with this command:

/etc/init.d/nagios start

If a configuration or syntax error in your configuration occurs, you can test the output and find out where it is by running this command:

/usr/local/nagios/bin/nagios -v /etc/nagios/nagios.cfg

Now you can try starting Nagios again. If you make configuration changes, the configuration must be reloaded. Do this use:

/etc/init.d/nagios reload

To access Nagios through the web, Apache must be started, or restarted if it already was running. This will also accept the new web configurations that may have been made:

/etc/init.d/apache2 start
/etc/init.d/apache2 restart
1Oct/090

Nagios – Installation, Configuration, and Use

Operating System: Debian Etch 4.0

An excellent resource to deploying Nagios is a book by Wolfgang Barth called "Nagios System and Network Monitoring" from No Scratch Press. It goes into nice detail on how to get the basics running or write your own plugins if you wish, plus many 3rd party tools to assist in monitoring.


Installation

First install the essential packages needed to compile and use Nagios as root.

apt-get install libgd-dev libssl-dev
apt-get install build-essential
apt-get install openssl
apt-get install apache2 mysql-server libmysqlclient-dev
apt-get install gcc make autoconf automake libgd ntpdate libperl-dev libnet-snmp-perl libdbd-sybase-perl

Create the users and groups that will be used to run Nagios.

groupadd -g 9000 nagios
groupadd -g 9001 nagcmd
useradd -u 9000 -g nagios -G nagcmd -d /usr/local/nagios -c "Nagios Admin" nagios

Add the Apache dameon user to Nagios group, which by default in Debian is www-data.

usermod -G nagcmd www-data

Now setup the initial directories and permissions where we'll keep the files. I chose to create /home/nagios because that's where I'll store the data files since the home partition is the largest on the server.

mkdir /usr/local/nagios/ /etc/nagios/ /home/nagios
chown nagios:nagios /usr/local/nagios /etc/nagios /home/nagios

Download the Nagios source and unpack.

cd /usr/local/src
wget http://superb-east.dl.sourceforge.net/sourceforge/nagios/nagios-3.x.x.tar.gz
tar zxvf nagios-3.x.x.tar.gz

Build the source including the location of the directories created previously. The sysconfdir is pretty standard, but localstatedir needs to have a bit more space if you have it.

./configure --sysconfdir=/etc/nagios --localstatedir=/home/nagios --with-command-group=nagcmd --with-perlcache --enable-embedded-perl

Observe configuration summary and make sure everything is OK.

*** Configuration summary for nagios 3.0.6 12-01-2008 ***:

General Options:
-------------------------
Nagios executable:  nagios
Nagios user/group:  nagios,nagios
Command user/group:  nagios,nagcmd
Embedded Perl:  yes, with caching
Event Broker:  yes
Install ${prefix}:  /usr/local/nagios
Lock file:  /home/nagios/nagios.lock
Check result directory:  /home/nagios/spool/checkresults
Init directory:  /etc/init.d
Apache conf.d directory:  /etc/apache2/conf.d
Mail program:  /usr/bin/mail
Host OS:  linux-gnu

Web Interface Options:
------------------------
HTML URL:  http://localhost/nagios/
CGI URL:  http://localhost/nagios/cgi-bin/
Traceroute (used by WAP):  /usr/sbin/traceroute

If the config looks OK, compile all:

make all
make install
make install-init
make install-commandmode
make install-config

This will allow Nagios to be started upon boot.

update-rc.d nagios defaults 99

Now download and install the latest batch of plugins so Nagios has something to run. Again, pay close attention to the configure statement so that the directories are the same as those specified for compiling Nagios itself.

cd /usr/local/src
wget http://superb-west.dl.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.x.tar.gz
tar zxvf nagios-plugins-1.4.x
cd nagios-plugins-1.4.x
./configure --sysconfdir=/etc/nagios --localstatedir=/home/nagios --enable-perl-modules
make
make check
make install

Test the ICMP plugin since it is a necessary component.

/usr/local/nagios/libexec/check_icmp 10.10.0.1

Configuration

Configure the Nagios web interface (Apache):

cd /usr/local/src/nagios-3.x.x
make install-webconf
/etc/init.d/apache2 reload

Create an htaccess file for simple authentication into the Nagios monitoring site.

cd /etc/nagios
htpasswd -c htpasswd.users nagios
chown www-data htpasswd.users
chmod 600 htpasswd.users

Be sure when you add other users that can login, you add them to the cgi.cfg as well.

htpasswd htpasswd.users otheruser

For easier readability and configuration, set configuration site with a unique name. This will help if you're monitoring multiple sites and will have many config files. Notice the primary and global config files are in /etc/nagios but the primary configurations that need editing can go under it. Nagios will recursively check all files the end with .cfg.

cd /etc/nagios
mv objects mysite

Third-party Utilities

These utilities make it easier to manage and gather information from Nagios, especially for reporting reasons.

NDOUtils

Download and install ndoutils for database storage of performance data.

cd /usr/local/src
wget http://superb-east.dl.sourceforge.net/sourceforge/nagios/ndoutils-1.4.x.tar.gz
tar zxvf ndoutils-1.4.x.tar.gz

Check to make sure you're using the right versions for compatibility.

fgrep CURRENT_OBJECT_STRUCTURE_VERSION nagios-3.x.x/include/objects.h
define CURRENT_OBJECT_STRUCTURE_VERSION        307     /* increment when changes are made to data structures... */
fgrep CURRENT_OBJECT_STRUCTURE_VERSION ndoutils-1.4.x/include/*/objects.h
include/nagios-2x/objects.h:#define CURRENT_OBJECT_STRUCTURE_VERSION        2
include/nagios-3x/objects.h:#define CURRENT_OBJECT_STRUCTURE_VERSION        307     /* increment when changes are made to data structures... */
./configure --sysconfdir=/etc
make
cd src
cp ndo2db-3x ndomod-3x.o log2ndo file2sock /usr/local/nagios/bin/

Prepare MySQL for ndoutils.

mysql> create database nagios_perfdata;
mysql> grant usage on *.* to 'nagios'@'localhost' identified by 'password' with max_queries_per_hour 0 max_connections_per_hour 0 max_updates_per_hour 0;
mysql> grant select, insert, update, delete on 'nagios_perfdata'.* to 'nagios'@'localhost';
mysql> flush privileges;
mysql> quit
cd /usr/src/ndoutil-1.4.x/db
mysql -u root -p nagios_level3 &lt; mysql.sql

Configure ndoutil for use.

cd /usr/local/src/ndoutil-1.4.x
cp config/ndo*.cfg /etc/nagios/
cd /etc/nagios

Modify /etc/nagios/ndomod.cfg:

output=/home/nagios/ndo.sock buffer_file=/home/nagios/ndomod.tmp Modify /etc/nagios/ndo2db.cfg: socket_name=/home/nagios/ndo.sock db_name=nagios_level3 db_user=nagios_dbuser db_pass=password

Test the ndoutil daemon and see if it runs.

/usr/local/nagios/bin/ndo2db-3x -c /etc/nagios/ndo2db.cfg
ls -l /home/nagios/ndo.sock srwxr-xr-x 1 nagios nagios 0 2009-01-02 15:02 /home/nagios/ndo.sock
kill -9 [ndo2db-3x pid]
rm /home/nagios/ndo.sock

Create an inet daemon for ndoutil.

cd /etc/init.d
cp skeleton ndo2db-3x
chmod 755 ndo2db-3x

Edit the new ndo2db-3x file:

#! /bin/sh
### BEGIN INIT INFO
# Provides:           ndo2db-3x
# Required-Start:   $local_fs $remote_fs
# Required-Stop:   $local_fs $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Example initscript
# Description:         This file should be used to construct scripts to be
#                          placed in /etc/init.d.
### END INIT INFO

# Author: Ryan Rosiek
#
# Please remove the "Author" lines above and replace them
# with your own name if you copy and modify this script.

# Do NOT "set -e"

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="ndoutil daemon startup for ndo2db-3x"
NAME=ndo2db-3x
DAEMON=/usr/local/nagios/bin/$NAME
DAEMON_ARGS="-c /etc/nagios/ndo2db.cfg"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
...
do_stop() {
   ...
   # Remove sock file
   rm -r /home/nagios/ndo.sock
   ...
}
...

Modify the Nagios config to start event broker module.

vi /etc/nagios/nagios.cfg
...
broker_module=/usr/local/nagios/bin/ndomod-3x.o config_file=/etc/nagios/ndomod.cfg

Start the services.

/etc/init.d/ndo2db-3x start /etc/init.d/nagios reload

Log into the database and make sure information is being written.

mysql -u nagios_dbuser -p
mysql> use nagios_level3
mysql> select * from nagios_objects where objecttype_id=2;

NRPE

NRPE makes it easier to run service checks on remote clients. First add it to the server.

cd /usr/local/src
wget http://localhost/nrpe-2.x.tar.gz
tar zxvf nrpe-2.x.tar.gz
cd nrpe-2.x.tar.gz
./configure --sysconfdir=/etc/nagios --localstatedir=/home/nagios --enable-ssl=no
make
make install

Note: SSL was not configured in this case because all monitoring traffic is internal and no access to this network is allowed

NSClient++

The NSClient++ is a great utility to run on Windows servers and for NRPE to check. Download latest client and unzip.

  1. Create folder C:\Program Files\NSClient
  2. Copy unzipped files into this folder
  3. Uncomment these lines in NSC.ini
;NRPEListener.dll
;CheckSystem.dll
;CheckDisk.dll
;CheckEventLog.d
;allowed_hosts= (under Settings)
;use_file=1
;allow_arguments=1
;port=5666
;use_ssl=0

Comment out:

;check_other=-H 192.168.0.1 -p 5666 -c remote_command -a arguments

In a command prompt:

cd \
cd "Program Files\NSClient"
NSClient++ /install