Table Of Contents

Previous topic

Implementation

Next topic

Monitor Types

Operation

A set of command line tools are provided for managing the Agent. The tools are in the system path if env.sh has been sourced correctly.

asagent

UNIX

The asagent command is used to start and stop the agent from the command line. By default the agent will daemonise itself and disconnect from the terminal. When the daemonise option is set to false however it will stay attached to the terminal and it can be stopped by using SIGINT or Ctrl-C to interrupt it. When running normally the agent can be stopped by sending SIGTERM (or SIGINT) to it. The --stop option is a convenient shortcut for this.

When the init.d script is installed this can also be used to start and stop the agent using the normal system tools.

All the settings documented in the Runtime Configuration appendix have a corresponding long option which can be passed in on the command line. On top of these the agent takes the following options:

--help
Show short usage information.
--stop
Stop the agent by sending SIGINT to the PID in the pidfile.
--status

Return the status of the agent, in accordance with the LSB init script convention:

0        asagent is running
1        asagent is dead and pid file exists
3        asagent is not running
4        status is unknown
--createdb

This option allows you to (re)create an empty database for asagent to work with. The database created by this will be determined by the store_uri setting.

When invoked in this mode asagent does not run as a daemon and will exit after creating the new empty database. Most settings other than store_uri and store_uri will be ignored.

-f --foreground
Run asagent in the foreground. By default asagent will daemonise, but using this option will keep it connected on the console with the log output also appearing on the console. This is an alternative to using the daemonise setting.
-l <level>
Define the level the agent will log at; this is a shorthand for –loglevel. Valid values for level are as specified in the Runtime Configuration section.

Windows

When installed on Windows a useful set of commands are provided via the Abilisoft Command Prompt. To use it open the Start menu and right-click on Computer and navigate to the Abilisoft.com program group. Select Abilisoft Command Prompt.

Note

If a User Account Control dialogue appears, click Yes to allow the command window to open.

The following commands are provided:

  • maql - Command line tool used to interact with agents, see the section maql.
  • ascrypt - Command line tool used to create security tokens.
  • asregex - Command line tool used to test regular expressions.
  • keyczart - Command line tool used to manage security token keys.
  • gather - Command line tool used to collect information to send with a support request, see the section for gather.py.
  • asagent - Run the agent in the foreground (not as a service).
  • python - Run the Abilisoft python interpreter.

Note

On Windows the asagent command only works with the following parameters; --help, -f, -l and --createdb.

On Windows platforms asagent is installed as a Windows service and can be started and stopped in the following ways:

Via the Computer Management Console

Open the Start menu and right-click Computer (on some Windows versions this item will be called My Computer). Click Manage to open the Computer Management Console. Under Services and Applications select Services. Select the service named asagent and click the link Start the service.

To stop or restart the agent follow a similar procedure but click the appropriate link in the Computer Management Console window.

Via the Command Line

Open the Abilisoft Command Prompt and enter:

c:\> sc start asagent

To stop the agent enter:

c:\> sc stop asagent

Note

If you believe the agent hasn’t started or isn’t operating as you expected then check the Windows Event Log which is visible in the Computer Management Console in System Tools/Event Viewer/Windows Logs/Application or look at the agent’s own logs in %AS_HOME%/logs.

Using Configuration Outside the Installation Directory

It is possible to run asagent so that it is not dependent on configuration stored inside /opt/abilisoft.com (keeping configuration here means it can be mistakenly deleted during agent software upgrades). The example below shows how the cfg_inifile setting can change the location the runtime configuration is sought for to the /etc/abilisoft directory:

$ asagent --cfg_inifile=/etc/abilisoft/asagent.conf

Note

The file referred to must exist at this location, it is not automatically created. Furthermore if you use the the init.d mechanism to start and stop the agent you must modify the init.d script to include the –cfg_inifile parameter when invoking the agent.

In the following example the manifest_uri setting changes the location the monitoring configuration is sought for to a web service called get_manifest running on the “foo.com” server:

$ asagent --manifest_uri=http://configs.foo.com/get_manifest

Or to a another part of the file system:

$ asagent --manifest_uri=file:///etc/abilisoft/manifest.xml

Running Multiple Instances

As mentioned in the name setting it is possible to run multiple instances of asagent on the same host. Since the name setting will change the default names of the configuration file, logfile, pidfile, and database it is easy to run another instance, e.g.:

$ asagent --name svcmon --createdb
$ asagent --name svcmon
I: asagent application initialising (1.2.3  r1234  19700101)
$ asagent --name svcmon --status
asagent (svcmon) running

In the above example the agent will subsequently use the following files:

Configuration:$AS_HOME/etc/svcmon.conf
Database:$AS_HOME/var/lib/asadb/svcmon.db
Pidfile:$AS_HOME/var/run/svcmon.pid
Logfile:$AS_HOME/log/svcmon.log

One issue to look out for when running multiple instances of asagent is that the AAPI ports used by one instance may clash by those required by another instance. To ensure the AAPI port requirements do not conflict between different agent instances, use suitable settings for xmlrpc_enabled, xmlrpc_addr and xmlrpc_port.

Running as an AgentX Subagent

It is possible to run asagent as an AgentX subagent. This means the agent will attempt to connect to a master agent listening on the configured port using the AgentX protocol. To do this you need to have an AgentX master agent running. By way of example we will describe how to run net-snmp as an AgentX master agent and then describe how to run asagent as an AgentX subagent.

To run net-snmp as an AgentX master agent, execute:

# /opt/abilisoft.com/thirdparty/snmp/sbin/snmpd -V -r -f -L -C -x /var/agentx/master --rocommunity="public 0.0.0.0" udp:1161

Most of the parameters are related to running net-snmp standalone in the foreground with suitable output for demonstration purposes. The important parameter for us is -x /var/agentx/master which tells net-snmp to listen on the socket /var/agentx/master for sub-agent connections (/var/agentx/master is the default for both net-snmp and asagent but the parameter is necessary the first time we invoke the net-snmp daemon to make sure the socket is created).

For SNMP invocations from clients, say a MIB browser, --rocommunity provides basic user access and will cause the snmp daemon to listen on all interfaces and accept connections from clients using the public community string. The chosen transport is udp and the daemon will listen on port 1161.

Note

You will probably want to connect a MIB browser to the running net-snmp daemon. If it is running on a remote server make sure the server’s firewall is configured to allow udp connections over port 1161.

Next we need to run asagent as an AgentX subagent, to do this execute:

# asagent --agentx_connaddr=UNIX:/var/agentx/master --agentx_registeroid=.1.3.6.1.4.1.26788

This will cause asagent to connect to the net-snmp master agent’s socket at /var/agentx/master. We also register an OID subtree (the one asagent will handle requests for) with the master agent using the agentx_registeroid setting. The default value of .1.3.6.1 would normally be fine but in our experience net-snmp doesn’t cope well with that as a subagent registration value so it is best to register the Abilisoft enterprise OID.

Note

We are demonstrating this using command line parameters but remember you can configure these settings in either the etc/asagent.conf file or by setting an appropriate environment variable.

When the agent successfully connects to the net-snmp master agent, logging similar to the following will be emitted in the asagent log file:

INFO     agentx.session    Opened session: Session1
INFO     agentx_subagent   Connecting to AgentX master agent on UNIX:/var/agentx/master
INFO     agentx.protocol   Registered with master agent: <AgentX-Register PDU: sid=5 pid=1583034273 subtree=1.3.6.1.4.1.26788 pri=127>

If asagent fails to find the net-snmp master agent (e.g. if the socket names you used for master and subagent didn’t match, if the master agent is not running or is not configured to listen for subagent connections) then logging similar to the following will be emitted in the asagent log file:

INFO     agentx.session    Opened session: Session1
INFO     agentx_subagent   Connecting to AgentX master agent on UNIX:/var/agentx/master
ERROR    agentx_subagent   Socket error connecting to AgentX masteragent scoket: [Errno 111] Connection refused

If all is well open your favourite MIB browser and load any required Abilisoft MIBs, they can be found in $AS_HOME/share/mibs. Test your configuration is working by navigating to MIB-II data provided by the net-snmp daemon:

_images/browse-netsnmp.png

Navigate to asaHostOverview to browse host health metrics:

_images/browse-asagent-host-mib.png

You can also navigate to asaStore to browse the agent’s store table contents:

_images/browse-asagent-store-mib.png

maql

maql is a command line tool that enables you to interact with MA either locally or remotely. MAQL can be used interactively or in “batch” mode using the -b parameter. Before connecting to an agent with MAQL the agent must be running and you must have valid credentials (as defined by <User> tags in the configuration). To start MAQL:

# maql

To see command line options:

# maql --help

In interactive mode the following is displayed:

Abilisoft MAQL Version 2.5.0 (brannigan.1) started: 2009/05/24 11:47:29
Platform: SunOS
Type "help" for help topics. Type "exit" to quit MAQL

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Secure mode is ON, passphrases sent to MA are encrypted
Type "help secure" for more information
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

MAQL>

In interactive mode the user is presented with the MAQL> command prompt. There is help for each command:

MAQL> help

MAQL commands (type "help <topic>" for more information):
=========================================================
EOF     check   getcfgall  loglevel  quit     set     status
Q       exit    help       manifest  savecfg  setcfg  time
SELECT  getcfg  host       q         select   show    uptime

Other Help Topics
=================
general  ACTION        STATUS       SAMPLE   ACTIONX
secure   OBSERVATIONX  OBSERVATION  SAMPLEX  PROCESS

MAQL> help host

usage: host [user@<hostname|ip>[:port]] [password]

Select a host running an MA to work with. You may specify a
hostname or IP address and a username.  You will be prompted
for the password if it is not specified.  An initial check is
made for MA availability and the uptime is printed.

Example: Specify an MA to work with and get its status:

MAQL> host admin@neptune
Password:
 neptune found (uptime: 23 Hours 14 Minutes 10 Seconds)
MAQL>
MAQL> status

    Hostname: neptune
          ip: 192.168.254.161
  Start Time: 2009/04/24 10:35:31
     Up Time: 23 Hours 14 Minutes 10 Seconds
 Config Path: /export/home/davec/mtmcore/etc/asagent.conf
  Config DTS: 2009/04/22 09:15:33
        %cpu: 0.5
         rss: 30010245
         vsz: 35945185

The primary purpose of MAQL is to explore the agent status, the data acquired, the observations made and the actions raised by it. You can also examine and modify the agent runtime configuration and invoke a monitoring configuration update check. As can be seen in the above example, any agent interaction must start with establishing contact with an agent. If it is present, the agent uptime is printed. If an agent is unavailable:

MAQL> host admin@neptune
Password:
ERROR: Connection failed: Failed to connect to agent.

Most MAQL commands have optional command parameters; MAQL will prompt you for the command parameters it requires if you do not specify them. Notice in the above example however the connection string used to specify an MA and user to connect as. If one is not specified you will be prompted for a user, a hostname (or IP address and optionally a port number).

MAQL> host
Host or IP: 192.168.2.155
User: admin
Password:

One important command “check” allows you to invoke a manifest update check in an agent:

MAQL> host admin@localhost
Password:
INFO: localhost:28001 found (uptime: 59 Seconds)
MAQL> check
Manifest update check requested
MAQL>

Note that this would be cumbersome for a large quantity of agents and much more scalable solutions are easily implemented either using the AAPI directly or using the API over XML RPC. Examples of how to do this can be found in: $AS_HOME/share/doc/asagent/examples/integration.

The most useful MAQL command is the select command. This allows you to use SQL style commands to get data out of the agent. The tables MA maintains that are available to query are (Table schemas are described in Table Schemas):

  • SAMPLE - Contains MA sample data.
  • OBSERVATION - Contains MA Observations.
  • ACTION - Contains MA Actions.
  • STATUS - MA status information, each row contains the information available to the status command and is collected periodically.
  • PROCESS - A snapshot of the host’s process table, built on demand.

So for example, one can get a list of processes currently executing on the host (The output is truncated for clarity):

MAQL> select * from process

1,0,root,0,root,0,20,1,59,0,0,1240331006.41,0.904387864,0.0,0.0,2203648,...
2,0,root,0,root,0,0,1,98,0,0,1240331006.41,0.000287908,0.0,0.0,0,0,pageo...
3,0,root,0,root,0,0,1,60,0,0,1240331006.41,1184.82680507,0.0,0.0,0,0,fsf...
7,1,root,0,root,0,20,12,59,7,7,1240331008.71,11.405826108,0.0,0.0,162447...
...

The default output format is CSV. It is possible to format the output in a tabular fashion:

MAQL> set pretty on
MAQL> select pid,exe,rss from process

pid                            exe                            rss
------------------------------------------------------------------------
1                              init                           2203648
2                              pageout                        0
3                              fsflush                        0
7                              svc.startd                     16244736
9                              svc.configd                    10706944
142                            syseventd                      3407872
153                            drd                            1646592
...

Or as XML, note the useful inclusion of the query statement that provided the xml data.

MAQL> set xml on
MAQL> select pid, exe, rss from process

<?xml version="1.0" encoding="utf-8"?>
<Result>
  <Query><![CDATA[select pid, exe, rss from process]]></Query>
  <Row>
    <Attr name="pid">1</Attr>
    <Attr name="exe">init</Attr>
    <Attr name="rss">2203648</Attr>
  </Row>
  <Row>
    <Attr name="pid">2</Attr>
    <Attr name="exe">pageout</Attr>
    <Attr name="rss">0</Attr>
  </Row>
  <Row>
...

Most SQL valid for a SELECT query is allowed (although queries are sanitised, you cannot DROP, UPDATE, INSERT and so on). You can use where clauses, nest queries, use aggregate functions and use SQL keywords like ‘LIMIT’ or ‘AS’. For example:

MAQL> select distinct facetvalue as 'Volumes' from SAMPLE where facetid like
     'volumeName%'

Volumes
------------------------------
/
/etc/svc/volatile
/export
/opt
/opt/abilisoft.com
/srv
/tank
/tank/install
/tank/ldoms
/tank/ldoms/scale
/tank/solaris
/tank/testzone

MAQL options can be preset in a configuration file. It attempts to read the configuration file in ~/.maql.conf and failing that in $AS_HOME/etc/maql.conf. Here is an example configuration file:

[options]
secure = on
pretty = on

This will switch on MAQL secure mode and pretty output by default which means one does not have to type (for example) set secure on and set pretty on every time MAQL is started.

MAQL also has a batch mode mechanism. One can specify a batch file on the command line and MAQL will read the commands and execute them in sequence. So, given a batch file called etc/maql.cmd with the following contents:

select * from sample limit 1
manifest
check

The following result is obtained. One can see the query, manifest and check command results:

# maql --batch etc\maql.cmd --host admin@localhost --password admin

cpu.overallCpu.200905251616400000,percentTotal,7.0,float
<Manifest xmlns:xi="http://www.w3.org/2001/XInclude"
          name="default"
          updated="2009/04/17 16:26:55"
          effectiveFrom="2007/01/01 00:00:00">
  <User name="admin" pwd="admin">
  ...
  </User>
</Manifest>
Manifest update check requested

XML RPC API

MAQL uses the agent’s API which is an XML-RPC API provided over HTTPS on port 8080. The API is disabled by default but can be enabled via the xmlrpc_enabled setting. You can also configure the API address and port with the xmlrpc_addr and xmlrpc_port settings respectively. When enabled the API is exposed over HTTPS by default, use the aapi_ssl to use HTTP transport instead.

You can easily write your own programs to interact with the agent’s AAPI, the methods and their parameters are described Table 14. Here is a simple example written in the Python programming language to make the agent check for an updated monitoring configuration:

>>> import xmlrpclib
>>> s = xmlrpclib.Server('http://neptune:8080')
>>> s.check('admin', 'admin')

To allow this invocation a user configuration would need to be defined in the agent’s monitoring configuration manifest as follows:

<User name=”admin_no_rpc” pwd=”admin”>
  <Feature name="ma.aapi.rpc_check">true</Feature>
</User>

The following sub-sections describe each AAPI method in detail. Note that for a feature named ma.aapi.rpc_XXX the corresponding method is called XXX. Some parameters have defaults (e.g. see the query method’s ‘pretty’ parameter) and do not need to be specified, for example the following is valid:

>>> s = xmlrpclib.Server('http://neptune:8080')
>>> s.query('admin', 'admin', 'select * from action')

However, if a default parameter is specified, any preceding default parameters must also be specified:

>>> s.query('admin', 'admin', 'select * from action', False, True)

check(user, pwd):void

Invoke a check for new monitoring configuration.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element

forcecheck(user, pwd):void

Invoke a check for new monitoring configuration. This differs from check in that this call will cause the agent to load the monitoring configuration regardless of whether it is detected as updated or not).

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element

query(user, pwd, query, pretty=True, pretty_xml=False, col_width=20):string

Query the agent store. Returns a string representation of the query result in the chosen format.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element
query string The SQL query string, e.g. select * from SAMPLE
pretty=True bool Formats the result into columns
pretty_xml=False bool Formats the result as XML
col_width=20 int Specifies the column width to use when pretty is True

manifest(user, pwd, resolved=False):string

Get current monitoring configuration. Returns a string containing the manifest XML.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element
resolved=False bool If true, all xincludes are expanded

setmanifest(user, pwd, manifest):string

Inject a monitoring configuration manifest into the agent. If successful this method returns the XML <Success/>, otherwise returns the XML <Error/> containing an error message.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element
manifest string The monitoring configuration manifest content.

setfragment(user, pwd, name, data):string

Save a monitoring configuration fragment using the name provided in the agent’s etc directory. If successful this method returns the XML <Success/>, otherwise returns the XML <Error/> containing an error message.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element
name string The file name to use when saving the fragment. This must include the .xml extension.
data string The fragment content

getfragment(user, pwd, name):string

Get a previously ‘set’ monitoring configuration fragment. If successful this method returns the XML <Success/>, otherwise returns the XML <Error/> containing an error message.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element
name string The file name of the fragment. This must include the .xml extension.

getcfg(user, pwd, item):string

Get a runtime configuration value. Returns a string representation of the value.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element
item string The name of the configuration setting, e.g. manifest_uri.

setcfg(user, pwd, item, value):void

Set a runtime configuration value.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element
item string The name of the configuration setting, e.g. manifest_uri.
value string New value for the configuration setting. Note that not all setting changes will take effect at runtime and that settings are not persisted between invocations unless the savecfg method is called.

getcfgall(user, pwd):string

Get all runtime configuration values. Returns all runtime configuration values in an XML format.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element

savecfg(user, pwd):void

Save runtime configuration updated with setcfg calls. This ensures AAPI RTC changes are persisted between agent restarts.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element

getloglevel(user, pwd):string

Get the log-level the agent is currently writing to its log file at. Returns the name of the log-level, e.g. DEBUG.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element

setloglevel(user, pwd, level):void

Set the level the agent should write to its log file at. Note that this takes immediate effect but does not persist between agent restarts.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element
level string The name of a log-level. Valid values are VERBOSE, DEBUG, INFO, WARNING, ERROR and CRITICAL. Note the VERBOSE log level is only available in special debug builds.

status(user, pwd):string

Get a summary of the agent’s status. Returns a simple string detailing hostname, IP address, start time, cpu etc.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element

uptime(user, pwd):int

Get the agent uptime in seconds. Returns a the uptime value in seconds. Note this method is a minimum requirement for an MAQL initiated session and as such so should be enabled if the user is to connect using MAQL.

Parameters Type Description
user string Username as specified in the <User> element
pwd string Password as specified in the <User> element

tools

The agent also ships with a tools directory inside $AS_HOME. This contains several small scripts that can be of use in some situations. Essentially these are small tools that haven’t been matured yet to proper binary “status”.

env.sh

When sourced this script will adjust environment variables like $AS_HOME, $PATH and $MANPATH which will enable to you run the agent and it’s tools in the current shell without having to modify the system environment in any way.

Usage:

$ . /opt/abilisoft.com/asagent/tools/env.sh

Changed in version 2.6.4: Previously this file was called mtmSetup.sh for legacy reasons, it will still be available under this name as a symlink for the entire 2.6.X series of releases.

dbgenv.sh

This script works exactly like env.sh but will also add environment variables that will produce more detailed information in the event of any problem occurring. When reporting issues it is recommended that you source this file rather than env.sh. This will enable Abilisoft Support to quickly solve issues.

New in version 2.6.4.

gather.py

This is a script that will gather all information required when reporting issues. It collects information about the asagent installation, it’s host operating system and dependencies as well as the configuration, database and logfiles. Normally it should be invoked as such:

$ $AS_HOME/tools/gather.py [options]

Even if the Python interpreter normally used by asagent is not available this script should still produce meaningful data when run with any other Python interpreter. In this case you can invoke it like this (here python can obviously be a pathname to a working python interpreter):

$ python $AS_HOME/tools/gather.py [options]

The result will be a tarfile which needs to be included with any issue reported.

-h, --help
Show short usage information.
-o <file>, --output <file>
Save the output in a specific file instead of a temporary created file.
--manifesturi <URI>
Normally gather.sh will try and read the MasterManifestURI setting from asagent.conf like asagent does. Sometimes this might not be correct or fail however, this option allows you to specify an explicit MasterManifestURI.

Note that on releases before 2.6.0 (calculon.0) the gather.py script was not executable and contained a sub-optimal #! line. On those versions there is a very simple wrapper script you can invoke:

$ $AS_HOME/tools/gather.sh [options]

launch-demo-snmptrapd.sh

Very simple shell script which will launch and SNMP trap receiver on the localhost. The default manifest.xml traps will be able to be received by this and they will simply be printed on stdout.

Usage:

$ $AS_HOME/tools/launch-demo-snmptrapd.sh

browse_agentx.py

This script allows you to look at what OIDs are available in the internal AgentX sub-agent shipped with asagent. It has three modes of operation allowing you to view the subtrees that get registered, to issue AgentX-GET requests and to issue a series of AgentX-GETNEXT requests starting at a certain subtree (also known as “walk”).

It can normally be invoked as such:

$ $AS_HOME/tools/browse_agentx.py [options]
-h, --help
Show short usage information.
-d, --dump
Dump out the registered subtrees to stdout. For each available context all subtrees that got registered by the subagent will be printed.
-g <OID>, --get <OID>
Perform an AgentX-GET request for a specific OID. The context used will depend on the browse_agentx.py --context option.
-w <OID>, --walk <OID>
Print all varbinds available below the OID specified in the MIB hierarchy, commonly known as “walking” the OID. The context used will depend on the browse_agentx.py --context option.
--sleep <TIME>
This tool needs to start the internal asagent AgentX subagent before it can query it. Once the subagent is started it needs to register it’s subtrees to the master agent started by this tool. Since it is impossible to know when the subagent has registered all the subtrees it knows about this tool just waits for a little while for the subagent to perform this registration. The default is to wait for 1 second but on a busy server this might be too slow and this option allows you to change this.
-c <context>, --context <context>

The AgentX context to use. AgentX allows subagents to register subtrees in several logically separated namespaces called a “context”. This options allows you to change the context to use.

The default is to use the '' context, that is an empty string as context, since this is the most common context required. However to also support no context you can specify --context '' to denote no context. Any other string will simply be the context denoted by that string.

-v, --verbose
Verbose logging of the tool. This will include details about starting and communication with the subagent.