Netmagis

Installation

Contents

• What's new
• Introduction
• Specifications
• Documentation
• Support
• Download
• Screenshots
• Demonstration
• Development
• License

Netmagis 2.2.x installation

Note: this installation documentation is for Netmagis version 2.2.x (x >= 3) If you are installing from a previous version of Netmagis, you should rather read upgrade instructions.

Program and file installation

Determining which packages are needed

The netmagis is divided into packages in order to install only what you need:

Packages are designed such as they may be installed on one unique server as well as each on its dedicated server, depending on your security requirements, performance considerations or site specificities.

• a large site may wish to separate all components:
  • the www package is installed on a central web hosting platform
  • the database package is installed on a central database cluster
  • the servers package is installed on the existing DNS and DHCP server, as well as on mail relay cluster
  • the detecteq package is installed on both the central syslog server and the central Radius server
  • the topo package is installed on a dedicated virtual server, for security reasons (access to the network equipements via the administration VLAN)
  • the metro package is installed on a dedicated physical server for performance and security reasons
• on the opposite, a small site may wish to run all packages on a unique virtual server.

The www and the topo packages must be installed on the same architecture (CPU and OS), even if the servers are physically distinct.

Installing from FreeBSD ports

At this time, the supported way to install Netmagis is via FreeBSD ports, which are provided in a separate archive (file netmagis-freebsd-ports-*.tar.gz associated with appropriate netmagis version).

Once the file is downloaded on a fairly recent FreeBSD system (minimum FreeBSD 8.3-RELEASE with up-to-date ports), use:

# tar xzvf netmagis-freebsd-ports-2.2.0.tgz -C /usr/ports/net-mgmt/

In order to install a netmagis package, just enter the following commands (just as well as any other port):

# cd /usr/ports/net-mgmt/netmagis-database
# make install

The provided ports automatically install required dependencies.

Installing from Debian/Ubuntu packages

Linux packages are provided for installing on:

• Ubuntu (minimum distribution is 11.10, Oneiric Ocelot)
• Debian (minimum distribution is wheezy - stable)

Append the following lines to your /etc/apt/sources.list:

deb http://netmagis.org/files/repo stable main
deb-src http://netmagis.org/files/repo stable main

If you are using an Ubuntu distribution, check that the "universe" section is enabled in /etc/apt/sources.list.

Now, update your package database:

# apt-get update

The next step depends upon your architecture:

• If your system is based on an i386 or amd64 architecture (i386 or x86_64 in "uname -i" output), you can install from binary packages (example for the database package):

  # apt-get install netmagis-database

  Since packages are not signed, you must manually answer "Yes" to the question "Install these packages without verification?".

• If your system is based on another architecture, you need to rebuild from source package:

  # apt-get install dpkg-dev
  # apt-get build-dep netmagis
  # apt-get source --build netmagis

  Once the packages are built, install them with gdebi (example for the database package and the pdp11 architecture):

  # apt-get install gdebi
  # gdebi netmagis-database_2.2-1_pdp11.deb

Installing from Netmagis source distribution

1. Download the Netmagis distribution file and unpack the archive:

   $ tar xzvf netmagis-2.2.0.tar.gz

   Alternatively, if you are adventurous, you may also fetch the current Netmagis sources directly from the gitlab repository. Provided that git is installed on your system, use this command:

   $ git clone git@gitlab.com:netmagis/netmagis.git

2. Install required dependencies. Please note that Tcl is a required dependency for all packages.
   • Linux (Ubuntu/Debian) note:

     # common and database
     aptitude -y install make git \
             postgresql-9.1 postgresql-pltcl-9.1 libpgtcl1.5 \
             tcl8.5 tcl8.5-dev tcllib pwgen

     # www
     aptitude -y install
             apache2 libaprutil1-dbd-pgsql \
             netpbm texlive-latex-base graphviz transfig

     # topo
     aptitude -y install \
             gcc fping rancid default-mta

     # metro
     aptitude -y install \
             libsnmp-session-perl libsnmp-perl libnet-snmp-perl librrdtool-oo-perl \
             libdbd-pg-perl rrdtool

     # ldap authentication
     aptitude -y install openldap-utils slapd

   • Linux (CentOS) note:

     If you are installing for a CentOS distribution, you need to install the Epel repository or to download each package not in CentOS repository (such as pwgen, tcllib, etc.) from http://dl.fedoraproject.org/pub/epel/6.

     Then, you can use yum to install dependancies:

     # common and database
     yum install tcl tcllib pwgen tcl-devel \
             tcl-pgtcl postgresql-server gcc \
             make postgresql-pltcl
     

     # www
     yum install httpd apr-util-pgsql mod_ssl

     # topo
     yum install rancid fping

     # metro
     yum install rrdtool-perl net-snmp-libs net-snmp-perl \
             perl-Net-SNMP net-snmp-libs perl-Digest-HMAC \
             perl-IO-Socket-INET6 perl-Crypt-DES \
             perl-Net-SNMP perl-DBD-Pg

     Then, change the top-level Makefile to suit your environnement:

     TCLSH = /usr/bin/tclsh
     TCLCONF = /usr/lib64/tcl8.5/tclConfig.sh
     

3. Choose an installation directory. You may use /local/netmagis/ for example. Netmagis's Makefile will install a hierarchy such as:

   .../etc/	various configuration files
   .../bin/	programs which should be called as an ordinary user
   .../sbin/	programs which should be called as the root user
   .../lib/	libraries, messages, and programs which are not meant to be called by humans
   .../share/	examples, default configuration files, database upgrade scripts, etc. NB: on Debian/Ubuntu, examples files are in /usr/share/doc/netmagis/examples/
   .../var/	files generated by topo or metro packages
   .../www/	Web application (installed by www package)

4. Modify the top-level Makefile variables to suit your choices:

   DESTDIR	Installation directory
   TCLSH	Full path name of tclsh interpreter
   TCLCONF	Tcl configuration script (example: /usr/lib/tcl8.5/tclConfig.sh on Linux or /usr/local/lib/tcl8.5/tclConfig.sh on FreeBSD)

5. Install the package:

   # make install-common

6. Install the desired package(s). For example, if you want to install the database, www and the utils packages, type:

   # make install-database
   # make install-www
   # make install-utils

That's it! You may now proceed to initial configuration.

Initial PostgreSQL configuration

On the host where you installed the database package, you should have a PostgreSQL server. The recommanded version is at least 9.1.

You have to check that your PostgreSQL installation allow connections (and authenticate users with password) from hosts which act as clients of the Netmagis database (all hosts where you installed a netmagis package). This is usually configured in file pg_hba.conf which is located in:

Netmagis relies on password authentication, even if the database is on localhost, so the "ident" method won't work. You should change the configuration in pg_hba.conf to look like this :

host all all 127.0.0.1/32 md5
host all all ::1/128 md5

Netmagis uses only one PostgreSQL user, without special privilege (except database creation privilege). You must create it (as the PostgreSQL administor user: pgsql on FreeBSD, postgres on Linux) with the PostgreSQL createuser program:

createuser --no-superuser --no-createrole --createdb --pwprompt nm

Even if you want to use a different database for the mac package, you can use the same user.

The Netmagis configuration file

The Netmagis configuration file is located in .../etc/netmagis.conf on each host where Netmagis is installed. The common package does not install this file (in order to leave your file intact if you previousely installed Netmagis), but it installs a copy in .../etc/netmagis.conf.sample. Copy it manually to .../etc/netmagis.conf, and edit it.

The following table list only directives you would want to modify for a fresh installation.

Directive	Description
dnsdb...	Information needed to connect to the PostgreSQL server, with an unprivileged user (see below).
macdb...	Information needed to connect to the PostgreSQL server for the mac package, with an unprivileged user. If you don't want to use a separate database, or if you don't want to use the mac package, use exactly the same values as for the the dnsdb... parameters.
ouiurl	URL to the IEEE OUI database (a default value is provided in the sample configuration file). This is necessary to populate the mac.oui table with Ethernet vendor prefixes when the mac database is created.
whoami	Command used to get login name of the user who is running the current program. It should display exclusively the login name. Default value is suitable for FreeBSD and Ubuntu.
rootusers	Login names of Netmagis users allowed to access the application even when it is in "maintenance mode".

Use the script netmagis-config (check your PATH) to test that Netmagis understands your modifications. The command:

$ netmagis-config dnsdbhost

dnsdbhost="myhost.example.com"

You can now proceed to database installation.

Database installation

PostgreSQL server should be initialized as described above. Netmagis should then be installed by providing .../etc/netmagis.conf directives:

Directive	Description
dbcopy	Name of a copy of the Netmagis database, generated every night (see the netmagis-dbmaint program in the database package). This copy lets you experiment and work on data without fear. If this directive is empty, the maintenance command will not create a copy of the database.
pwgen	Command line to generate a random password
crypt	Command line to encrypt a password, compatible with the mod_auth_pgsql Apache module

• If you are upgrading from a previous WebDNS or Netmagis installation, please see the upgrade instructions.
• If you are installing Netmagis for the first time, after you configured the netmagis.conf file, run the command

  $ .../sbin/netmagis-dbcreate

  This command:
  • creates the Netmagis database (it fails if the database is already created)
  • creates the MAC database if it is different from the main Netmagis database (it fails if the database is already created)
  • creates the SQL schemas, tables, functions and triggers
  • provides default values for Netmagis directives
  • creates Netmagis users listed in rootusers directive (in .../etc/netmagis.conf file)
  • creates a Netmagis user called nobody
  • assigns all Netmagis users a random password, and displays them on standard output
  Users are created in the Netmagis database, including the internal authentication schema.
  • if you use an LDAP authentication, rootusers users should exist in your LDAP directory.
  • if you plan to use the internal authentication database, note the generated passwords as they will be needed in the Web application installation
• Unless you are upgrading from previous versions, you should import your data into the database. Doing so is described in the section below.

  Since this is a long task, if you want to test your installation, you may try to import example data. You may do so by issuing the command to load the multi-views example (you can load the no-view example instead):

  $ cd .../share/examples/netmagis/with-views
  $ sh ./run-all.sh

  You can later destroy and re-create the database with your own data.

• Install the sample crontab (.../share/examples/netmagis/crontab.database) in your default cron system. This crontab can be run by any unprivileged user.

  FreeBSD users: if you install this crontab in the global /etc/crontab, please provide a user name just before the command name.

  The .../sbin/netmagis-dbmaint command performs some maintenance tasks:

  • make a hourly backup of database in the directory named in the dumpdir directive from .../etc/netmagis.conf
  • copy the current database in a copy named after the dbcopy directive
  • performs a PostgreSQL vacuum operation
  • remove some old entries in Netmagis internal spools

Netmagis database is now installed. You may now install the www package.

Web application installation

To store user accounts, the Web application can use either an existing external LDAP directory or the internal PostgreSQL database.

netmagis.conf directives
Directive	Description
passwdurl	URL which should be displayed to let users change their password, if you use an external LDAP authentication. Leave empty if you use internal PostgreSQL authentication.
hostcmd	Path to the host command
pwgen	Command line to generate a random password (not used for LDAP authentication)
crypt	Command line to encrypt a password (not used for LDAP authentication)

• A default Apache configuration file is provided in ..../share/examples/netmagis/httpd.conf. Adapt this file (notably for your authentication choice) and include it in your Apache configuration.
• Direct now your Web browser to the http://yourhost/netmagis/index, and connect with your login name. If you use the internal Netmagis PostgreSQL authentication, use the password generated by the netmagis-dbcreate command, during the database creation phase.

  You should see a main menu such as this one.

• If you are using LDAP or CAS authentication, go into the Admin menu (on the left vertical bar), then to the Application parameters menu.

  In the "LDAP authentication parameters" section, adapt values to your LDAP directory. Then select the LDAP item in the Authentication method menu (in the "General parameters" section). Then, click on the Submit button to register your new configuration.

  Select any item in the left menu to check that your authentication parameters are correctly configured.

• If you are using internal (PostgreSQL based) Netmagis authentication, change your password (unless you want to keep the generated one).

  Change also your name (as well as other account informations) with the Admin / Internal Auth / Modify account menu.

Congratulations! The Web application is now installed.

Import your existing data

Your next task is to feed your database with referential data (users, groups, networks, etc.) as well as your existing DNS zones if any.

Configuring Netmagis

Documentation about common tasks in Netmagis configuration is provided in a separate page.

Utils package installation

The utils package is optional: it contains commands to add, delete or modify individual hosts, as well as DNS zone prologues. These commands are designed to be used wth the Unix shell.

Installation is very simple: only common netmagis.conf directives to connect to the database are needed.

Notice, however, that this file (including password to connect to the database) must be readable to all users of these scripts. This may be a security risk if you make this file readable for all users, but you can also control access to the configuration file and to the scripts based on a group policy.

The provided commands are:

• dnsaddhost fqdn addr view: adds the specified host with the IPv4 or IPv6 address
• dnsaddalias fqdn-alias fqdn-host view: adds the specified alias to an existing host.
• dnsdelhost fqdn view: deletes the specified host
• dnsdelip addr view: deletes an IP address. If this is the last, deletes the host.
• dnsmodattr fqdn view key val key val ...: modify an existing host. Possibles keys are:
  • mac: value is the MAC address
  • hinfo: value is the exact name of the the machine type, as listed in the Modify machine types administrator menu. or in the dns.hinfo SQL table

  • respname: name of user responsible for this host
  • respmail: mail address of user responsible for this host
  • comment: comment associated to this host
  • dhcpprofile: value is the exact name of the DHCP profile, as listed in the Modify DHCP profiles administrator menu. or in the dns.dhcpprofile SQL table
  • ttl: value is the specific TTL value for this host, in seconds, or empty string to remove the specific value
  • sendsmtp: value is 0 to remove the right to emit mail with unauthenticated SMTP, or 1 to add the right.
• dnsreadprol zone view: displays on standard output the DNS zone prologue specified by its domain name.
• dnswriteprol zone view file: use the named file as the zone prologue. Since this operation may have a great impact, manual confirmation is required.

With all these commands, you have to provide a view name. If you don't use multiple views, just use the name "default".

Note that users using this script must have a Netmagis account with the same login as the Unix one.

Servers package installation

The servers package must be installed on your main BIND server, on your DHCP server, and on your mail router, depending on the features you want.

On all servers

On all servers, you will have to configure .../etc/netmagis.conf with the following directive:

BIND server

On your main BIND server, provide values for the following directives:

Once you configured .../etc/netmagis.conf file, run (as the user who have access to the zonedir directory and to the zonecmd command):

$ .../sbin/mkzone -n -v

You can force generation of some zones by using the Admin / Force zone generation. This sets a flag in the dns.zone SQL table, which is read by the mkzone to find modified zones.

Test now a real zone generation by using:

$ .../sbin/mkzone -v

Each zone file will be named after the "Name" column in zone edition.

If you are using multiple views on different DNS servers, you can use the -w view option to limit zone generation to a single view. By default, the mkzone program generates zone files for all views.

If everything is ok, you can use the sample .../share/examples/netmagis/crontab.bind to install the periodic zone generation for the user.

An usual convention about the SOA serial number is to use a number of the form "yyyymmddnn" where yyyymmdd is the current date and nn is a sequence number for this date. Netmagis uses this convention when possible, but if there is more than 100 modifications in a day, it will simply increment the value. Our algorithm will automagically synchronizes again with the convention when possible. This way, you can run the mkzone script every minute without fear.

FreeBSD users: if you install this crontab in the global /etc/crontab, please provide a user name just before the command name.

ISC DHCP Server

Netmagis is designed to aliment a central ISC DHCP server. If you want to use this functionnality on other networks, you need to activate DHCP relay in your router configurations.

The Netmagis can also handle failover configuration in order to provide a redundant DHCP service based on 2 ISC DHCP servers.

On your DHCP server, provide values for the following directives:

Once you configured .../etc/netmagis.conf file, run as root:

$ .../sbin/mkdhcp -n -v

Test now a real DHCP file generation by using:

$ .../sbin/mkdhcp -v

If you are using multiple views, you need to supply a specific view for which you are generating a DHCP file, with the -w option:

$ .../sbin/mkdhcp -v -w internal

If everything is ok, you can use the sample .../share/examples/netmagis/crontab.dhcp to install the periodic DHCP file generation. You should run the DHCP file generation soon after the DNS zone generation (such that new host names are not generated in DHCP file before they are known in the DNS).

FreeBSD users: if you install this crontab in the global /etc/crontab, please provide a user name just before the command name.

Mail relay cluster

Netmagis can be used to generate mail routes for you central mail router. Routes are generated following "mail roles", whose modifications are enabled individually for each allowed domain in the Admin / Modify users and groups menu.

Once you configured .../etc/netmagis.conf file, run as root:

$ .../sbin/mkmroute -n -v

Test now a real file generation by using:

$ .../sbin/mkmroute -v

If you are using multiple views, you need to supply a specific view for which you are generating mail routes, with the -w option:

$ .../sbin/mkmroute -v -w internal

$ .../sbin/mkmroute -v -w internal

If everything is ok, you can use the sample .../share/examples/netmagis/crontab.mroute to install the periodic file generation.

FreeBSD users: if you install this crontab in the global /etc/crontab, please provide a user name just before the command name.

Authenticated SMTP filtering

If you use authenticated SMTP, you know than some hosts (copiers, embedded measurement machines, etc.) may not be able to use authenticated SMTP. Some exceptions must be done.

Netmagis helps you handle these exceptions, by providing a special flag to each host. Right to activate this flag is enabled for each group in the Admin / Modify users and groups menu.

To activate this functionnality, some prerequisites are needed on your mail infrastructure:

• Authenticated SMTP message submission is enabled on TCP port 587
• Unauthenticated SMTP access is enabled on TCP port 25
• All your routers restrict use of port 25, such as internal hosts can only open connections to mail submission hosts
• Your mail submissions hosts accept connections on port 25 only from authorized hosts.
  A configuration file for a *BSD PF (packet filter) is provided as an example in .../share/examples/netmagis/pf.conf.sample. You will notice that the smtpf table is read from a file, generated by Netmagis.

With this infrastructure, provide values for the following directives on your mail submission hosts:

Once you configured .../etc/netmagis.conf file, run as root:

$ .../sbin/mksmtpf -n -v

Test now a real file generation by using:

$ .../sbin/mksmtpf -v

As with mkmroute, you can use the -w view option to specify a particular view.

If everything is ok, you can use the sample .../share/examples/netmagis/crontab.smtpf to install the periodic file generation.

FreeBSD users: if you install this crontab in the global /etc/crontab, please provide a user name just before the command name.

Congratulations! You have installed the servers package. The Netmagis is ready to be used, for its core functionnality.

Detecteq package installation

The topo package needs to know when the configuration of a network equipment has been changed. These changes are detected via:

• syslog for Cisco and Juniper equipments
• radius accounting for HP equipements. In this case, since HP equipments are unable to report configuration changes via Syslog, the hack is based on Radius accounting which detects user logout (which do not strictly imply a configuration change).

Thus, the detecteq package must be installed on your central syslog (syslog, syslog-ng, etc.) server and/or your central radius server, depending on your equipments. Since modification events are sent to the database, the detecteq package does not need to be installed on the same server as the topo server.

Network equipment configuration

In order to automate configuration fetching from a central server, a unique user should be defined on every network equipment. Let us call it conf. This user should have the right to execute the show configuration commmand (or similar), which means it must have the "admin" privilege.

Moreover, at least on HP equipments, there should be another distinct user (let us call it modconf) which should be able to modify the configuration in order to change interface characteristics. This second user is needed for Radius to distinguish regular configuration fetching from configuration modification.

Syslog configuration

• On Cisco and Juniper equipments, you need to configure a syslog server if not already done. For example:

  Equipment	Example
  Juniper	system {     syslog {        host syslog-server.example.com {            any notice;            facility-override local2;        }     } }
  Cisco	logging syslog-server.example.com logging facility local2

• Configure .../etc/netmagis.conf for access to the Netmagis database
• Update your syslog configuration file and use a line such as:

  local2.* |exec /usr/local/sbin/detectconfmod

  Adapt this line to your syslog facility and the detecteq installation path. You can also use other syslog daemons such as syslog-ng if they provide the "pipe to program" functionnality.

  Note: on Linux (Debian/Ubuntu), the default syslog program (rsyslogd) does not support the "pipe to program" functionnality. An alternative is to install the syslog-ng package and configure it with lines such as:

  source remote {
      udp(port(514));
  }

  filter myequip {
      netmask ("10.20.30.0/24");
  };

  destination mylogs {
      file ("/var/log/myeq.log");
  };

  destination detectconfmod {
      program ("/usr/local/sbin/detectconfmod");
  };

  log {
      source(remote);
      filter(myequip);
      destination(detectconfmod);
      destination(mylogs);
      flags(final);
  }

• Be sure to keep hostnames in generated logs. You should get lines such as (example on a Cisco switch):
  

  Dec 23 11:52:26 adm-sw %SYS-5-CONFIG_I: Configured from console by joe on vty0 (192.168.1.1)

Freeradius configuration

• On HP equipements, configure Radius accounting using the following command:

  Equipment	Example
  HP	radius-server host radius.example.com key secret acct-port 1813

• Configure your existing Freeradius using the SQL accounting module and the example files provided in .../share/examples/netmagis/radius/ directory.
    sql {
      database = "postgresql"
      driver = "rlm_sql_${database}"
      server = "mydatabaseserver.example.com"
      login = "nmdetecteq"
      password = "mysecret1234"
      radius_db = "netmagis"
      sqltrace = no
      sqltracefile = ${logdir}/sqltrace.sql
      num_sql_socks = 5
      connect_failure_retry_delay = 60
      lifetime = 0
      max_queries = 0
      sql_user_name = "%{User-Name}"
      nas_query = ""
      authorize_check_query = ""
      authorize_reply_query = ""
      authorize_group_check_query = ""
      authorize_group_reply_query = ""
      accounting_onoff_query = ""
      accounting_update_query = ""
      accounting_update_query_alt = ""
      accounting_start_query = ""
      accounting_start_query_alt = ""
      accounting_stop_query = " \
          INSERT INTO topo.modeq (eq,login) \
          ( \
            SELECT rr.name || '.' || domain.name, \
            '%{User-Name}' \
            FROM dns.rr, dns.rr_ip, dns.domain \
            WHERE rr_ip.idrr = rr.idrr AND \
              rr.iddom = domain.iddom AND \
              rr_ip.addr = '%{NAS-IP-Address}' AND \
            '%{User-Name}' NOT IN ( \
                SELECT login \
                FROM topo.ignoreequsers) \
          ) \
          "
      accounting_stop_query_alt = ""
      group_membership_query = ""
      postauth_query = ""
    }
  
• As described above, the Radius accounting server needs to distinguish regular configuration fetching from configuration modification. This is done using the topo.ignoreequsers table which contains login names of the users to ignore.

  The default user is named conf. If want to change names of users which perform read-only (i.e. fetch) operations, issue the following SQL commands with psql:

  DELETE FROM topo.ignoreequsers;
  INSERT INTO topo.ignoreequsers VALUES ('joe');
  INSERT INTO topo.ignoreequsers VALUES ('bill');

Topo package installation

One of the most innovative parts of Netmagis is the topo package, which provides a complete infrastructure to fetch and analyze network equipments configuration files and modelize the network as a graph. This allows for automatic generation of network L2 and L3 maps, equipment port configuration, as well as interactions with the metrology package.

At this time, Juniper, Cisco and HP equipments (switches and routers) are supported.

Relations between topo and www packages

You have the choice to run the topo and www packages on the same host or on distinct hosts.

For security reasons, one would often use a separate server to run the topo package. This server should be located within the administration VLAN of equipments, if you have one.

The interface between the topo package and the rest of Netmagis is:

• the ability to send mail from the topo server

• the ability for the www package to access data extracted from the network graph built on the topo server: the topo server pushes (via a SSH connection) the file containing the built graph, and the www server runs extract commands on demand.

  This means that the topo package must also be installed on the www server to provide these commands.

  This also means that the architecture of the two servers must be compatible (CPU and OS) since the graph is a binary file.

The rest of this section describes the topo package installation.

On the topo server

The first part is the configuration of the topo server.

• The first step of installation is to configure the rancid software (prerequisite of the topo package) on the topo server.

  Detailed information can be found in the README coming with the rancid package. However, here are the minimal steps to a quick configuration:

  • Check that the rancid user is created
  • Choose a unique "group" name for all your network equipments. The topo package uses the name mygroup by default. The name must be configured in the file rancid.conf (location is system dependant), in the line beginning with LIST_OF_GROUPS. While you are here, notice the BASEDIR directory name.
  • Configure a mail alias rancid-groupname (in /etc/aliases or equivalent, don't forget to run the command newaliases or equivalent) to receive notifications when rancid detects modifications.
  • Configure the ˜rancid/.cloginrc file with authentication information. For example:

    add method * {ssh}
    add password myrouter {pw} {enablepw}
    add password sw-* {anotherpw} {anotherenablepw}

    See the cloginrc(5) manual page for details about this configuration file.

    Be sure that this file belongs to the rancid user and has mode 0600.

  • Initialize the directory tree by running (as the rancid user):

    rancid-cvs

    This program is located in /usr/local/libexec/rancid/rancid-cvs on FreeBSD and in /usr/lib/rancid/bin/rancid-cvs on Linux (Debian/Ubuntu).
  • In order to test your rancid installation, name some equipments in the router.db file, which should be located in $BASEDIR/group-name/router.db. See the rancid README for instructions.

    This file will be overwritten by the topo package with information from the Netmagis database.

  • Test your installation by running:

    rancid-run

    The test is ok if you received a mail with the equipment configuration, which is also stored in the $BASEDIR/group-name/configs/ directory.
  • The rancid README says to run rancid-run periocally with cron. Do not do this: the topo package will call rancid when needed.

  Notice: this is not related to Netmagis installation, but you should consider installing a Web interface to CVS (CVSWeb) to ease tracking of your equipement configuration history.

• The topo package must know which equipments are on each side of a network link. Therefore, you have to add information in equipment configurations: a link name (or number) prefixed with L, bracketed with "<" and ">", and added to the description of the interface on each side of the link:

  Equipment	Example
  Juniper	ge-0/1/0 {     description "blah blah <L233>; }
  Cisco	interface GigabitEthernet0/1     description blah blah <L233>
  HP	interface A1     name "blah blah <L233>"

  You are advised to use the Web application (Topo / Link number menu item) to get and register link numbers. This way, you keep track of used link numbers, and link numbers are ensured to be unique.

  Ports connected to individual hosts must be marked with a "<X>" to indicate a terminal node in the graph.

  A complete description of the topo configuration is available on a separate page.

• Configure .../etc/netmagis.conf on the topo server:

  netmagis.conf directives
  Directive	Description
  rancidcmd	Path to the rancid-run command
  ranciddir	Rancid group directory (usually $BASEDIR/group-name/) where are located: the router.db file which lists all polled equipments the configs/ directory which contains all configurations fetched by rancid
  sendcisco, sendhp, sendjuniper	Commands used to propagate new port configuration on an equipment, if you choose to enable the functionnality of letting users modify interface configuration.
  fpingcmd	Command used to launch fping, in order to test equipment before sending commands to change port configuration
  logger	Command used to send a message to syslog
  eqvirtdir	Directory used for "virtual equipments". Each file is a specially crafted configuration which represent one equipment whose configuration cannot be fetched by rancid
  eqgendir	Directory used to store results of analysis of rancid fetched configurations
  topobindir	Directory where all topo programs are searched
  topograph	Path name of the graph built during topo analysis
  topocpgraph	Parameters (user@host:remote-path) used to copy the graph built during topo analysis to the www server. Leave empty if the topo and www servers are the same host.

• Change the ownership to the rancid user:

  # chown -R rancid .../var/netmagis

• As the rancid user, run the following command:

  $ .../lib/netmagis/topo/anaconf

  You will undoubtly notice a large number of warning that you will correct with time. If no fatal error has been encountered, you will normally find in .../var/netmagis/network.graph the modelization of your network. This is a binary file.

  To check that things are ok, you can use the commands (which all take the binary graph as standard input):

  • dump the graph in clear text:

    .../lib/netmagis/topo/dumpgraph -a < .../var/netmagis/network.graph

  • display all interfaces characteristics of an equipment:

    .../lib/netmagis/topo/extracteq -a equipment < .../var/netmagis/network.graph

• Be sure that the detecteq package is properly installed on your central syslog server and/or your central radius server.
• Run the topographd and the toposendd daemons via the provided startup scripts:

  FreeBSD	/usr/local/etc/rc.d/topographd /usr/local/etc/rc.d/toposendd
  Linux	copy or link XXXXXXXXXXX .../etc/rc.d/topographd.sh and .../etc/rc.d/toposend.sh to /etc/init.d directory

• If you have choosen to install the topo and www packages on different servers, you have to configure the SSH connection from the rancid user on the topo server to a user on the www server.

  This SSH connection is used to copy the network graph computed by the rancid user on the topo server to the www server. The destination user on the www server will own the copied graph, which must be readable by the CGI scripts running under the Apache user. This destination user may be, for example, the Apache user (called www, www-data, nobody, etc., see the "User" directive in Apache configuration files). Let us call it the w user. Make sure this w user (on the www server) has a home directory and a valid shell.

  Then, as the rancid user on the topo server, create a SSH keypair (without pass-phrase):

  ssh-keygen

  Copy the public key (˜rancid/.ssh/id_rsa.pub) to the ˜w/.ssh/authorized_keys on the www server.

  Make sure that the topocpgraph configuration parameter is initialized with a line such as:

  topocpgraph w@www.example.com:.../var/netmagis/network.graph

  Warning: the path supplied here is the path of the graph on the server, which should match the topograph directive on the www server.
• If you have choosen to install the topo and www packages on the same server, make sure that the topocpgraph directive is empty.

On the www server

You need now to come back to the server which hosts the Web application.

• If you have choosen distinct www and topo servers, you must first install the topo package (on the www server) in order to use programs which extract data from the network graph. Then, you have to configure the following directives in .../etc/netmagis.conf on the www server:

  netmagis.conf directives
  Directive	Description
  topobindir	Directory where all topo programs are searched (on the www server)
  topograph	Path name of the graph copied onto the www server.

• Direct your web browser to the Netmagis application, and go to the Admin / Application parameters menu. In this menu:
  • activate the topo package
  • change the "Default domain" value: the domain name of your network equipments
  • change the "Topo mail sender" value: you may use an invalid address such as automatic mails generated by the topo server don't receive replies.
  • change the "Topo mail receiver" value: topo mails will be sent to this address rather than to the genuine rancid-groupname alias you've previousely defined.
  After activation of the topo package, you will see a new Topology entry in the left menu.
• Make sure that all your network equipments (and the default domain) exist as hosts (and domain) in the Netmagis database.
• Input your network equipments (as fully qualified domain names) and your VLANs via the Admin / Modify equipments and Modify Vlans menus.
• You can monitor the topographd and toposendd daemons with the Admin / Topod status menu.
• If you want users to be able to modify interface configuration within the Web application, enter the Admin / Modify users and groups menu, and input regular expressions in the Permissions on network equipments section.

Configuring Topo

The Topo configuration documentation has all information to complete the Topo configuration.

Congratulations! You've configured the topo package. You can now consult your equipments, your network maps, and even modify interfaces of your equipments if you enabled this feature!

Metro package installation

The Metro package provides metrology data collection on equipments. Data collected include:

• traffic
• ip<->mac associations (ARP tables)
• port<->mac associations (Forwarding tables)

At this time, all kinds of equipments are supported for traffic and ip/mac collection, provided they run a standard SNMP MIB-2 agent. For port/mac collection, Cisco, Juniper and HP equipements are supported.

The metro application is made up of two main programs:

• metrocreatedb which creates metrology sensors based on the metrology collection points defined in the Netmagis database, resulting from equipement configuration fetching by the topo package.
• metropoller which performs data collection for each sensor

The first part of this section describes the configuration of the metro package.

• Configure .../etc/netmagis.conf on the metro server:

  netmagis.conf directives
  Directive	Description
  metrodatadir	Main data directory, where reporting, logs and rrdtool databases are stored (in sub-directories)
  rrdtool	Pathname to the rrdtool command
  rrddb	Directory where rrdtool databases are stored. Reserve enough disk space for this directory. It is usually a sub-directory of metrodatadir but it can be a separate directory if you need more disk space. FIXME: remove and keep only metrodatadir
  sensorsurvival	Grace period (in days) before sensor removal. This is needed when a sensor is no longer updated by topo (this can be caused by a volontary removal or a bad configuration of the equipment). So for safety reasons, the polling is really stopped on this sensor only after this grace period.
  pollergroups	FIXME: provide description of directive first, and explanation after. Sensors sharing the same characteristics can be put in the same group: the running time is short, so running them in the same group allows a high level of parallelization ; for example, the traffic collection sensors are usually put in the same group the sensors are specialized and the SNMP collection takes more time We recommand to start with the grouping provided in the default configuration and refine the configuration afterwards. You must declare all your groups with the directive "pollergroups". Names of the different groups are separated by a ",". FIXME: groups should be separated by a space instead of a comma
  gpopt_<pollergroupname>	The gpopt_* directives define which sensor types are associated with each group defined with the directive "pollergroups". If no sensor type is specified, all sensor types will be executed by the poller group. If you want that a poller group collects only some of sensor types, write the list of type names separated by a "," to the "gpopt_GROUP" directive. Example: gpopt_GP2 ipmac,portmac.cisco If you want that a poller group collects all sensor types except "ipmac" and "portmac.cisco", write a "!" before each type name of sensor separated by a "," to the "gpopt_GROUP" directive. Example: gpopt_GP1 !ipmac,!portmac.cisco
  gpnbproc_<pollergroupname>	Set the number of processes launched simultaneously for each poller group. The data collection task is shared between the different processes which run in parallel. You can raise the number of processes to reduce the total execution time for all sensors.
  ifspeed_<ifaceprefixname>	Set the speed (in Mbit/s) associated to the prefix name of an interface type. This configuration is useful to tune the maximum values of rrd databases when the SNMP ifspeed values collected on the interfaces are not reliable. For example: ifspeed_Dot11Radio 300 will limit the maximum speed for all interfaces called "Dot11Radio..." (cisco Aironet radio interface) to 300 Mb/s.
  pollerlogfacility	Set the syslog facility for data collectors. Modify your syslog configuration accordingly. Example for syslog-ng:     filter f_local0 { facility(local0); };     destination d_pollerlog { file(".../metro/log/poller.log"); };     log { source(s_src); filter(f_local0); destination(d_pollerlog); };
  majdblogfacility	Set the syslog facility for sensor creation. Modify your syslog configuration accordingly. Example for syslog-ng:     filter f_local1 { facility(local1); };     destination d_majdblog { file(".../metro/log/majdb.log"); };     log { source(s_src); filter(f_local1); destination(d_majdblog); };
  pollerlogpt	Set the default syslog level. (FIXME: change directive to pollerloglevel)
  metrohost	Set to the server name hosting the metro package ; if metro is installed on the same server as the www package set it to "localhost".

• Activate the creation of new sensors and collection of data. For this, you must create a user that will run all the metro programs and make this user the owner of all the metrodatadir and rrddir directory trees. Let us call this user m.

  Install the sample crontab (.../share/examples/netmagis/crontab.metro) in your default cron system. This crontab should be run by the user m.

  After the fisrt execution of .../bin/metrocreatedb, polling can start.

• Allow access from the www server to the metro webservice: NB: the www host must be trusted by the metro server. The access to the webservice is not authentified.
  Copy the file .../share/examples/apache-metro.conf to your apache configuration directory and adapt it to your environment. The line starting with Allow from must have the IP address of the server running the www package. If www and metro both run on the same host, just put localhost.

The metro package is now ready to work. You can create sensors in the equipment configurations:

• To activate a traffic sensor on an equipment interface, the only thing to do is to add information in the interface description on the equipment, as you have done with topo package.

  This information is the string M concatened with the name of the sensor. You can use any unique name for a sensor. For example, Netmagis authors use the link number (used with the topo package) associated with the name of the equipement (since a link number is present on each end of the link), but you can use your own convention.

  For example:

  Equipment	Example
  Juniper	ge-0/1/0 {     description "core link <L233 M233.eqname>"; } ge-0/1/1 {     description "edge port <X Meqname.desc>"; }
  Cisco	interface GigabitEthernet0/1     description core link <L233 M233.eqname> interface GigabitEthernet0/2     description edge port <X Meqname.desc>
  HP	interface A1     name "core link <L233 M233.eqname>" interface A2     name "edge port <X Meqname.desc>"

• To collect ip-mac and port-mac session, you need to enable the mac package in the application configuration in the Admin web interface.
  To activate an ipmac sensor in your equipement configuration, you have to add information in the SNMP sysLocation description.

  This information is just the letter I, bracketed with "<" and ">" added to syslocation string.

  Important: The sensor works only if routing is enabled.

  Equipment	Example
  Juniper	snmp {     location "hi, I'm here <I>"; }
  Cisco	snmp-server location hi, I'm here <I>
  HP	snmp-server location "I'm here <I>"

• To activate a portmac sensor in your equipement configuration, it is the same process is an ipmac sensor; just use <P> (instead of <I>).

  Important note: if you want activate both ipmac and portmac sensors on the same equipment, insert the string "<I P> in the sysLocation description.

Metro data descriptions

The Metro package generates saveral datas into the following files or directories :

• ../var/netmagis/metro/poller/
  This Directory contains the following files :
  • metro.sensor : list of parameters of all sensors for data collection. This file is generated automatically by metrocreadb.
  • metro.quarantine : lists all the sensors which are no longer updated by topo. The sensors are definitively removed after a grace period.
  • metro.plugins : this file is reserved to be manually updated in order to execute plugins or sensors which are not automaticaly activated.
• ../var/netmagis/metro/rrddb/
  The rrdtool databases are stored in this directory.
• ../var/netmagis/metro/log/poller.log
  You can find in this log file all the messages generated by sensors collection and plugins execution.
• ../var/netmagis/metro/log/syncmetro.log
  All the informations about the creation, deletion of sensors are in this log file.
• ../var/netmagis/metro/report/
  This directory contains the results of the ipmac and portmac collection before storing them in the mac database.
• ../var/netmagis/metro/cache/
  This directory contains the caches files created by the sensors collection. By example, the (ip,interface name,SNMP interface index) relation.
• ../var/netmagis/metro/lock
  This Directory contains the locks files created by some scripts (metrocreatedb, plugin-ipmac or plugin-portmac) in order to avoid multiple executions of the same script in the same time. Important : If a crash of a script occurs, the lock file must be removed.