Quickstart

This quick start guide will explain the basics on how to install, configure and run the Universal Probe (UP). More detail is provided in subsequent sections of this document.

Installing UP

UP is distributed as a tar archive, one for each supported platform. Basic installation is very simple, just ensure you have the correct tar archive for your platform and then execute:

$ tar -xzof up_donbot.1_x86-ubuntu10.04.tar.gz
$ cd up_d.1_x86-ubuntu10.04
$ ./install.sh
I: Host platform: x86-ubuntu10.04 (use --platform if incorrect)
I: Installing up...
I: done!

This will install UP into the default location of /opt/abilisoft.com/up, binaries are in /opt/abilisoft.com/up/bin.

Running UP

It is now possible to start up by invoking it directly:

$ /opt/abilisoft.com/up/bin/up
up application initialising (7.1 (donbot.1)  r14146  20130510)

However since no listeners, chains or dispatchers have been configured up will not do anything at this point (basic configuration steps are explained later in this section). To stop up:

$ /opt/abilisoft.com/up/bin/up --stop
I: Sent SIGTERM to PID=1279
I: Waiting for up daemon to stop . done

Rather than typing the path to the up binary each time, a shell script is provided for convenience. Source the script as follows:

$ . /opt/abilisoft.com/up/tools/env.sh
Following variables where set/modified:
...

This will update the current environment so that only up needs to be entered, e.g:

$ up
up application initialising (7.1 (donbot.1)  r14146  20130510)
$ up --stop
I: Sent SIGTERM to PID=6426
I: Waiting for up daemon to stop . done

Configuring a Listener

A Listener is the mechanism up uses to gather data from a particular source, for example, an SNMP trap source or a digital telephone exchange, which it then injects into its rules engine for processing by a particular rule chain.

Note

UP can be configured with as many listeners (of the same and different types) as you like.

To configure a Listener you must add a listener definition to a configuration file called /opt/abilisoft.com/up/etc/listeners.xml. The example below demonstrates how to configure a simple SNMP Listener:

<?xml version="1.1" encoding="UTF-8"?>
<listeners>
  <snmp id="trap">
    <chain>trap</chain>
    <synthevents>true</synthevents>
    <listen></listen>
    <port>162</port>
  <snmp>
</listeners>

An SNMP Listener enables up to receive SNMP traps from an SNMP trap source. Obviously the SNMP trap source must be configured to send traps to the interface and port up is listening on. The example above configures one SNMP listener named trap which will listen on all IPv4 and IPv6 interfaces on the default SNMP port (i.e. port 162).

Incidentally both the <listen/> and <port/> tags are optional, if omitted the default values shown in the example are used. An empty <listen/> value means “listen on all interfaces”. Setting the <listen/> tag to a specific IP address will restrict the interface that this listener will receive traps on.

Note

The “id” attribute is of the type xsd:ID as defined in the XML standard. You cannot have duplicate values of xsd:ID in an XML document and therefore you must ensure each listener you configure is uniquely named.

The configuration also specifies the <synthevents/> tag, setting it to true. This will cause the listener to generate synthetic events. Synthetic events are injected into the rules engine and represent significant listener events like “started” and “stopped”, or when certain error conditions arise. The combination of synthetic events raised is listener dependent, refer to each listeners’ documentation for details. However, all listeners can raise the start, stop and error synthetic events.

Most importantly, the configuration defines the <chain/> tag, which specifies that all events received by this listener (and synthetic events generated by it) are to be injected into the trap chain. This is the entry-point to the rules processing for all incoming events for this listener and the rule chain is where we will write rules that decide how incoming events are handled.

A full description of available up listeners can be found in the Listeners section.

Configuring a Dispatcher

Dispatchers are the destinations to which the up rules engine (executing a rule chain) will send processed data. A dispatcher is configured in similar fashion to up listeners using the file /opt/abilisoft.com/up/etc/dispatchers.xml:

<?xml version="1.1" encoding="UTF-8"?>
<dispatchers>
  <reef id="reef">
    <uri>http://localhost/reef/</uri>
  </reef>
  <email id="mail">
    <sendmail/>
  </email>
</dispatchers>

The example above demonstrates how to configure two dispatchers:

• ‘’reef’’ - An Abilisoft Reef dispatcher. This makes it possible to send availability event data to the Reef event management server.
• ‘’mail’’ - An email dispatcher. This makes it possible to send data to one or more e-mail recipients.

Reef Dispatcher

The Reef Dispatcher only requires a single URI as its configuration. The URI specifies the HTTP endpoint that the Reef server is listening on. Refer to the Reef Administrators’s Guide or contact your Reef Server administrator to ascertain what this value should be.

Email Dispatcher

The Email Dispatcher is configured to use the local MTA. It is also possible to configure the email dispatcher to use an SMTP server, please refer to the Email Dispatcher section for more details.

The example configuration above makes these two dispatchers available to the rule chains using their names: reef and mail. A full description of available up dispatchers can be found in the Dispatchers section.

Writing a Rule Chain

Referring to the listener configuration in the example above you will recall that the trap listener was configured to use the trap chain. Rule chains are written in text files using the .chain extension and are kept in the up configuration directory, so for example the example chain below should be saved to /opt/abilisoft.com/up/etc/trap.chain:

# -*- coding: utf-8; mode: python -*-

"""Very simple rule chain to handle SNMP traps

This rule chain handles incoming SNMP events from the ``trap`` SNMP
listener.
"""

import up

def rule_synth_stop(event):
    """Send an email for stop synthetic events"""
    if event.synth and event['name'] == 'stop':
        out = up.EmailEvent(event)
        out.to = 'noc@example.com'
        out.subject = '{id} listener stopped'.format(id=event.listener.id)
        out.body = """Warning,

Received a stop synthetic event for the {lstnr.id} {lstnr.kind}
listener.  Please take appropriate action.
""".format(lstnr=event.listener)
        mail_dispatcher = up.getdispatcher('mail')
        mail_dispatcher.send(out)
        return None
    else:
        return event


def rule_synth(event):
    """Drop any remaining synthetic events"""
    if event.synth:
        return None
    else:
        return event


def rule_community(event):
    """Drop events from unknown communities"""
    if (event.listener.kind != 'snmp'
            or event['community'] not in ['public', 'private']):
        return None
    else:
        return event


def rule_create_outgoing_reef(event):
    """Create the outgoing reef event

    This rule creates the basic outgoing reef event and stores it
    on the incoming event under the "out" attribute.  It uses the
    snmpTrapOID varbind value to create the reef_key which is
    probably overly-simplistic.
    """
    out = up.ReefEvent(event)
    out['reef_key'] = event['snmpTrapOID']
    event.out = out
    return event


def rule_severity(event):
    """Set the severity of the reef event"""
    if event['community'] == 'private':
        event.out['reef_severity'] == 'major'
    else:
        event.rout['reef_severity'] == 'minor'
    return event


def rule_dispatch(event):
    """Dispatch to the Reef manager"""
    dispatcher = up.getdispatcher('reef')
    dispatcher.send(event.out)
    return None

Before discussing each logical section of this example rule chain we will take a closer look at the inbound data (the event parameters passed into each rule).

Inbound Data

The inbound data is data injected by a listener into a rule chain and represents a data item received by a listener. What constitutes a data item depends on the type of listener. For example, for an SNMP Trap listener it would represent a single received SNMP trap. All incoming data are up.Event instances and are passed into a rule case as the one and only parameter (event parameter in the example above). These objects behave like read-only dictionaries (we discuss dictionaries later) with some extra attributes:

data:The real dictionary containing the key-value pairs of the inbound data. Typically you shouldn’t need to modify the inbound data but in the rare cases that you do, you can use this attribute. listener:An up.ListenerProxy instance describing the listener from which this data was received. This only has two attributes: id, which is the id defined in listeners.xml and type which is the type of the listener, e.g. snmp. peer:The socket address of the peer. For IPv4 addresses this is a tuple of (host, port). For IPv6 addresses this is (host, port, flowinfo, scopeid). For some events (e.g. synthetic events) the peer will be None.

You can assign your own attributes to incoming data objects, a commonly used pattern to communicate information between different rules (e.g. an outgoing data object).

Preamble (lines 1 to 9)

Line 1 is a comment and normally optional, however it is best practice to include it. Firstly it tells up that the chain file is encoded using the UTF-8 character set. This becomes important when you want to use non-ASCII characters. UTF-8 is the default character set if not specified. Secondly this line is also recognised by many text editors, ensure the correct encoding is always used.

Note

The rule chains are written using the Python programming language syntax. The second part of the comment line is recognised by many editors and prompts them to display the code with syntax highlighting making it more readable.

Lines 3 to 7 constitute a document string (doc string for short) and are useful for documenting the purpose of the chain. The last line (line 9) is where we import the UP Rule Chain API, making it available for use within the rule chain. Usage of the Chain API will be described in more detail in subsequent sections.

Rule cases (line 11 onwards)

The remainder of the chain file defines a set of rule cases. Each rule case is a python function prefixed with rule_ that takes a single parameter event (the parameter name event is unimportant, when writing a rule case you can give the parameter any appropriate name, e.g. data) e.g:

def rule_synth_stop(data):

Any functions whose names are not prefixed with rule_ are ignored by the rules engine. Rule cases are executed by the rules engine in the order they are defined, starting with the first rule function defined, for each data item a listener injects. Once a rule case has processed the parameter passed to it, it can decide whether to allow chain processing to continue (using the next rule case), halt or continue processing in another chain.

The listener injected data item (we’ve called it event in this example) is a data object with attributes. The attributes are referenced using a dot notation, e.g. event.synth references the event’s synth attribute which can have a value of True or False. The injected data has a range of attributes, some dependent on the source of the data (i.e. which listener originated it) and some common amongst all listener generated data. These are documented in the relevant sections of this manual.

Additionally, the listener injected data item can behave like a “dictionary” object (i.e. a map of key-value pairs) so you can access data values by using a key as follows:

event['community']

Here we are referencing the value stored against the community key. The injected data has a range of supported key-values, some dependent on the source of the data (i.e. which listener originated it) and some common amongst all listener generated data. These are documented in the relevant sections of this manual.

rule_synth_stop (lines 11 to 26)

This rule case is the first to be executed when a listener that defines the chain with its <chain\> tag injects data. Points to note:

• The rule case has its own doc string describing the rule’s purpose (line 12).
• We check to see if the inbound event is a synthetic stop event (line 13). Data items injected by a listener have certain attributes that can be accessed to help processing, all listener injected data have the synth attribute.
• If the event is synthetic we use the Chain API create an outbound data item, one that can be consumed by a particular dispatcher. In this example we create an EmailEvent called out that we will pass to the Email dispatcher - because we want to warn the operations staff that for some reason a listener has stopped. The EmailEvent is constructed using the inbound synthetic event we are currently processing.
• Lines 14-22 build the necessary email fields (to, subject and body).
• Recall that we had configured an e-mail dispatcher called mail. On line 23 we use the Chain API to get a reference to the mail dispatcher.
• On line 24 we call the dispatcher’s send method and send our EmailEvent.
• On line 25 we end processing. Whenever we use the keyword return the rules engine will decide whether to pass the processing on to the next rule case or not. Returning None will ensure processing will stop for this data item. Returning the event data (like we do on line 27 when the inbound data is not a synthetic stop event) the event is passed to the next rule case.

Note

The title case of the None string is important, make sure you use the string None and not “none” or “NONE”.

rule_synth (lines 29 to 34)

It is best practice to leave 2 blank lines between rule cases functions. This rule case accepts data returned from the previous rule case. If we get this far we know now that the event is not a synthetic stop event. We’ve decided we’re not interested in any other synthetic events passed from the listener so if it is some other kind of synthetic event we will “drop” it. That is, cease rule processing for this data item. As we know, we do that by returning None from the rule case. Otherwise if the event is not synthetic we return the event in order to give the next rule a chance to look at this data item.

Note

We don’t have to process synthetic events, we could simply configure the listener that sends data to this chain with <synthevents>false</synthevents>. However using synthetic events is good practice, ensuring that anomalous conditions are captured.

rule_community (lines 37 to 43)

This chain has been written to process SNMP traps injected by the SNMP trap listener. Therefore we would like to drop events that were not injected by an SNMP trap listener, or if they were, have an appropriate community string (i.e. public or private):

event.listener.kind != 'snmp'

is quite self explanatory. All inbound data has a listener attribute that represents the source listener, which in turn has a kind attribute that identifies the kind of listener that injected the data. Remember any listener can be configured to name any chain so it’s good practice to checks the origin of the data and prove it’s appropriate to your processing. The following:

event['community'] not in ['public', 'private']

is a Python idiom for comparing a variable with a list of possible values. The statement above evaluates to true if the community key-value of the injected data is equal to neither of the strings ‘public’ or ‘private’. Using idioms is good practice as it makes your rules more readable.

rule_create_outgoing_reef (lines 46 to 57)

This rule case:

• Creates a ReefEvent object using the inbound event data called out
• Creates a unique key value for the outbound Reef event
• Payloads out onto the inbound event. This is a neat trick that allows us to create an outbound event early on in the rule processing and then enrich it over several rules. It can be passed from rule to rule by “piggy-backing” it onto the inbound event which is then passed to the next rule.

The section describing the Reef Event dispatcher describes how Reef Events can be employed to generate useful availability events that will be correlated and de-duplicated by the Reef Server and then displayed in the Reef UI with a an appropriate key, severity, label, description, automatic expiry times and so on. Reef related fields are supported by reef_ prefixed dictionary keys. For example reef_key can be set to drive the event correlation function in the Reef server that the dispatcher sends the ReefEvent to.

rule_severity (lines 60 to 66)

We’ve decided that all traps that have a community string value of private should have a major severity assigned, while anything else should have a minor severity assigned. Then processing is passed on to the last rule.

rule_dispatch (lines 69 to 73)

def rule_dispatch(event):
    """Dispatch to the Reef manager"""
    dispatcher = up.getdispatcher('reef')
    dispatcher.send(event.out)
    return None

This rule performs the actual event dispatch to the Reef server destination configured in the dispatchers.xml file by calling send passing in the ReefEvent we created in the rule_create_outgoing_reef rule and pay-loaded onto the event being processed. We return None to cease processing even though there are no more rule to process. By doing this we are telling the rules engine we have covered all cases.

When the rule processing for an inbound data item gets to the last rule, and the last rule does not return None, up will log a warning message to its log file. We call this data falling off the end of the chain and is not a desirable situation. It means data was injected but the chain didn’t know what to do with it. If you get such a log message then you should review you rule processing and make sure you capture the unexpected case, even if you simply “drop” the event.

Running the example

Make sure up is not running and update and saved your listener.xml and dispatcher.xml configurations in the /opt/abilisoft.com/up/etc directory. Additionally create a suitable /opt/abilisoft.com/up/etc/trap.chain file following the example given. Make sure your Reef server is up and running and correctly configured. Start up.

Make sure you have configured your trap sources to send SNMP traps to the host and port up is listening on. When traps arrive you will see them represented in the Reef UI. If you want to simulate a trap source to test your configuration you should use a trap sending utility like Net-SNMP.