Configuring and Using ITA Event Management System

Configuring and
Using the Dell
OpenManage™ IT
Assistant Event
Management System
Enterprise Systems Group (ESG)
Dell OpenManage™
Systems Management
Dell White Paper
By Ross Burns
August 2001
Contents
Introduction ................................................................................................................................... 3
How Alerts Reach IT Assistant .................................................................................................. 4
Event Population in OpenManage IT Assistant ..................................................................... 6
Event Categories, Types, and Sources ...................................................................................... 8
Creating User–Defined Events in IT Assistant ..................................................................... 11
Custom Event Setup: User Scenario ................................................................................... 12
Event Filters and their Configuration in IT Assistant ......................................................... 16
Event Actions and their Configuration in IT Assistant ...................................................... 20
Event Logging and its Configuration in IT Assistant.......................................................... 24
Events that IT Assistant Generates ......................................................................................... 26
Event Management and the IT Assistant Database ............................................................. 27
Conclusion ................................................................................................................................... 30
Tables
Table 1: Dell Agent Protocol Support ....................................................................................................... 4
Table 2: Events pre-populated in IT Assistant ......................................................................................... 6
Table 3: Database tables associated with ITA’s EMS .............................................................................27
Figures
Figure 1: Event Categories Dialog in ITA showing both Event Categories and Event Types .......... 8
Figure 2: Event Types Dialog in ITA showing Event Sources Defined for that Type ........................ 9
Figure 3: SNMP Event Source Configuration Dialog in ITA showing a Pre-populated Event
Configuration ......................................................................................................................................12
Figure 4: Filter Configuration Dialog in ITA showing a Minimum Filter Configuration ................17
Figure 5: Configure Filter Actions Dialog with Four Actions Configured .........................................22
Figure 6: Event Logging Configuration dialog from IT Assistant 6.1 .................................................25
August 2001
Page 2
Dell Enterprise Systems Group
Section
1
Introduction
In demanding customer environments, computer uptime means everything.
Having a computer go down for any length of time can mean a loss in
productivity and/or revenue. It is critical that computer systems consistently
operate in a healthy state and, when a system happens to develop a problem, the
appropriate system administrator be alerted as quickly as possible to avert any
downtime, no matter where he/she may be. Beginning with version 6.0, Dell
OpenManage™ IT Assistant possesses the features necessary to perform
immediate alerting via email, page, or other means when a monitored event is
received. This paper discusses the details of configuring IT Assistant’s new Event
Management System (EMS) to provide quick and accurate alerting.
For this paper, it is recommended that the user have a minimum installation of
IT Assistant v6.0 with Service Pack 1 applied, or has installed IT Assistant
version 6.0.1 from the Dell OpenManage Apps CD. IT Assistant version 6.1, to be
released in Q3 2001 calendar year, provides additional EMS enhancements,
which will be mentioned throughout this article. This paper assumes a basic level
of familiarity with using IT Assistant in a networked environment. Many
concepts here build on the IT Assistant’s User’s Guide.
August 2001
Page 3
Dell Enterprise Systems Group
Section
2
How Alerts Reach IT Assistant
Dell OpenManage IT Assistant (ITA) supports the reception of Desktop
Management Interface (DMI) indications and Simple Network Management
Protocol (SNMP) traps, both of which can be generically called “events”. Due to
the still-developing standard of the Common Information Model (CIM) event
model, ITA does not support CIM events at this time, but plans to do so in the
future. For ITA to receive DMI indications, it must register with the remote
system to subscribe to any events it must receive. ITA does this through a
Remote Procedure Call (RPC) to the remote system, and will automatically do
this when it discovers a system that supports the DMI protocol. See Table 1 for
the versions of Dell instrumentation that support the DMI protocol for
management. ITA’s Network Monitoring Service handles all reception of events
and dispatching of alerting.
For ITA to receive SNMP traps, the user must do some additional setup. Every
computer system that ITA is to monitor or manage must be configured to send
its traps to IT Assistant. Whether it be Novell® Netware®, Linux®, Microsoft®
Windows®, or any other OS capable of running an SNMP service, that operating
system’s SNMP service will need to have at least two pieces of information
configured for each destination sends its traps to: a community name and a
hostname (or ip address). While it is beyond the scope of this paper to describe
how to setup traps on the more common operating systems, this information can
be readily found in the respective operating system’s documentation and on
many Internet sites. Keep in mind that a restart of the respective SNMP service is
usually required for any changes to take effect. Another caveat about SNMP is
that it relies on the Universal Datagram Protocol (UDP) transport mechanism to
deliver its events, and many routers are setup to block these packets by default.
If it is suspected that ITA is not receiving traps as it should, any routers inbetween the source and destination should be checked for the allowance of UDP
packets. See Table 1 for those versions of Dell instrumentation that support the
SNMP protocol for management.
Table 1: Dell Agent Protocol Support
Dell Instrumentation
Dell Hardware Instrumentation
Package (HIP) 3.x
Dell OpenManage Server Agent
(OMSA) v4.0 – 4.3
August 2001
Supports DMI
Supports SNMP
Supports CIM
Yes
Yes1
No
Yes2
Yes3
Yes2,3
Page 4
Dell Enterprise Systems Group
Dell Instrumentation
Dell OpenManage Server Agent
(OMSA) v4.4
Dell OpenManage Client
Instrumentation (OMCI) v5.x,
v6.0
Dell Remote Assistance Card
(DRAC) v2.x Agent
Dell OpenManage Array
Manager (OMAM) Agent v2.5+
Supports DMI
Supports SNMP
Supports CIM
No
Yes
Yes3
Yes
No
Yes
No
Yes
No
No
Yes
No
1Via
Intel DMI-to-SNMP Mapper; N/A on Microsoft Windows 2000.
Windows only.
3Optional at agent install time (some versions).
2Microsoft
If ITA receives both DMI and SNMP events from the same managed computer
system, it is possible that duplicate events can be received for the same failure or
warning. Because of the different data structures of DMI indications and SNMP
traps, it can be nearly impossible to compare the two events and throw the
duplicate away, and ITA does not attempt to do this. However, ITA will perform
limited filtering of the same exact event that is of the same protocol in order to
filter out repetitive events from the same failure – this will be discussed in
further detail later in this paper.
August 2001
Page 5
Dell Enterprise Systems Group
Section
3
Event Population in OpenManage IT
Assistant
ITA contains a repository, or database, where it stores known events that it may
receive from agents out on the network. Out of the box, ITA has pre-populated
this event repository with events generated by various Dell agents, as well as
other agents - see Table 2 for a list of these agents. For a user to create an event
filter comprising that event, or to even see a comprehensible message of what the
event indicates, it is necessary for ITA to have that event in its database so that it
can recognize the event. When ITA does recognize an event it has received, it
will format the message of the event to a comprehensible message and check for
any user-configured filters where this event is included. The user can control the
message formatting to provide more or less information, down to a per-event
basis. However, if an event comes in that ITA does not recognize, ITA will not
ignore the event. ITA will dump the raw contents of the event to the ITA alerts
window. For DMI this is the vendor-specific message field concatenated with
the vendor specific data field; for SNMP this is the specific trap id, generic trap
id, enterprise oid, and all varbind variables contained in the trap. As will be
discussed later, ITA supports user-defined events that will allow ITA to
recognize events that are not present in its pre-populated repository.
Table 2: Events pre-populated in IT Assistant
Agent
Dell OpenManage Client
Instrumentation (OMCI)
v5.x, v6.0, v6.1
Dell Hardware
Instrumentation Package
(HIP) v3.x
Dell OpenManage Server
Agent (OMSA) v4.0-v4.2
Dell OMSA v4.3-v4.4
Dell Remote Assistant Card
(DRAC) v2.x
Dell Remote Service Card
(DRSC) v1.0
Dell OpenManage Array
Manager (OMAM) v2.5
Dell OMAM v2.6 – v3.0
August 2001
Events Supported
in ITA 6.0
Events Supported
in ITA 6.0.1
Events Supported
in ITA 6.1
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No1
No1
Yes
Yes
Yes
Yes
No
No
Yes
Yes
Yes
Yes
No1
Yes2
Yes
Page 6
Dell Enterprise Systems Group
Agent
Dell PERC / PERC2 / PERC3
Adaptec CIO / CIO4
Symbios
Qlogic
Intel NIC Instrumentation
Broadcom NIC
Instrumentation
NuView ClusterX
Veritas ClusterX
DMTF Generic
Hewlett – Packard server
agent (s).
Compaq server agent(s).
1
Events Supported
in ITA 6.0
Yes
Yes
Yes
Yes
Yes3
Events Supported
in ITA 6.0.1
Yes
Yes
Yes
Yes
Yes3
Events Supported
in ITA 6.1
Yes
Yes
Yes
Yes
Yes3
No
No
Yes
No
No
Yes4
No
No
Yes4
Yes
Yes
Yes4
Yes
Yes
No5
Yes
Yes
No5
New agent version added new events – these new events are not yet supported in this ITA version.
2 Events
only present on an initial install of 6.0.1 from the Dell OpenManage Applications CD; installing ITA
Service Pack 1 does not update event support.
3 DMI
only at this time.
4 Support
for SNMP traps from the DMI-to-SNMP mapper is limited.
5 These
events were removed in ITA 6.1; performing an upgrade from 6.0.x and choosing to preserve the
database will retain these events.
August 2001
Page 7
Dell Enterprise Systems Group
Section
4
Event Categories, Types, and Sources
For the user to select which events are important, some sensible ordering of the
events must be presented. Much like other management applications supporting
event management, ITA arranges its database of registered events in a
hierarchical fashion. At the top of the hierarchy are event categories, at the
middle are event types, and at the bottom are event sources; complexity and
detail increases as the hierarchy is traversed from top to bottom. This hierarchy
can be accessed through the Event Categories link under the Configuration Menu
on the ITA Blue Bar. Event Categories consist of distinct logical/physical groups of
components that make up a typical computer system and/or computing
environment, see Figure 1. Each category is general by nature and can span
events from several agents.
Figure 1: Event Categories Dialog in ITA showing both Event Categories and
Event Types
August 2001
Page 8
Dell Enterprise Systems Group
Under Event Categories are Event Types – see Figure 2. Event types correspond to
individual events that an agent might send. For example, an agent monitoring
environmentals for a computer system will likely keep tabs on components that
influence and measure the environmental conditions of the system, namely
temperature probes and fan probes. If a fan inside the computer fails, that agent
will send a fan failure event. Subsequently, after the fan fails, the temperature
may begin to rise, first passing a temperature warning threshold (temperature
warning threshold event), and then possibly surpassing a temperature critical
threshold (temperature critical threshold event). Each of these events is an event
type under the Environmental category in ITA.
Figure 2: Event Types Dialog in ITA showing Event Sources Defined for that
Type
Within event types are Event Sources. It is possible to have one agent support
events through two different protocols, as Dell’s OpenManage Server Agent
(OMSA) does: it supports both DMI indications and SNMP traps. In addition, it
is possible to have multiple versions of the same agent present on a managed
network, such as Dell’s OMSA and Dell’s older Hardware Instrumentation
Package (HIP) agent. It is also possible for a different agent, such as Dell’s
August 2001
Page 9
Dell Enterprise Systems Group
Remote Access Card (DRAC), to send that same type of event (such as a
temperature warning threshold event), although for a different monitored
component, as Dell’s OMSA would. All of these possibilities lead to the creation
of event sources. An event source generally represents an event from a particular
agent sent via a specific protocol, such as a temperature warning threshold
SNMP trap emitted from a Dell OMSA agent. Event types are meant to collate
event sources of the same type, whether they may be from different protocols,
different agents, or even different versions of the same agent. The principle is
that a temperature warning threshold event is the same type of event, even if it
comes from different sources.
August 2001
Page 10
Dell Enterprise Systems Group
Section
5
Creating User–Defined Events in IT
Assistant
An Event Management System (EMS) would not be very useful if it did not allow
the user to define his/her own events. ITA fully supports custom user-defined
events in the way of SNMP traps or DMI indications, however the user should
have some background knowledge of the respective protocol before attempting
to setup a custom event. The bulk of the setup is done in the event source
window. This is where protocol-specific knowledge is needed. An overview of
the SNMP Event Source Configuration dialog and how ITA recognizes SNMP traps
is as follows (see Figure 3):

Event Source Name - This is a user-defined (arbitrary) name selected by the
user to identify the agent the trap comes from.

Format String – A text message that is shown to the user if that event is
received from the agent. This field should contain an extrapolated message
from the MIB for that event. In addition, certain variable substitutions can be
performed to provide additional information in the message. These
substitutions are denoted by a ‘$’ symbol in front of them. For a list of
possible substitutions, select the Help button while in the Event Source
Configuration dialog in ITA.

Enterprise ID - This field must be present and is matched to the trap’s
Enterprise OID.

Specific Trap ID - This field is optional1; if filled-in, it is matched to the
trap’s specific trap id.

Generic Trap ID – This field is optional1; if filled-in, it is matched to the
trap’s generic trap id.

Severity – This field is not matched to the trap. The user needs to select a
severity that closely matches the severity definition in the MIB in which the
trap is defined, or that reflects the view of the user configuring the event.

Severity Configuration by Value – Becomes active if By Value is selected for
the Severity field. Note: This option should only be used if there is one trap
definition that could indicate more than one severity – this is very rare! An
example would be that an agent sends out a “temperature event” where
It is strongly recommended to fill-in all fields used to match an event – once ITA finds
the first match, it stops looking for additional matches, which can lead to undesirable
results if some fields are omitted.
1.
August 2001
Page 11
Dell Enterprise Systems Group
message and severity are variables (known as varbinds in SNMP lingo)
contained in the trap, meaning that they could change, even though the
Enterprise ID, Specific Trap ID, and Generic Trap ID stay the same.
Figure 3: SNMP Event Source Configuration Dialog in ITA showing a Prepopulated Event Configuration
Custom Event Setup: User Scenario
In an effort to better explain why and how a user might setup a custom event, a
fictional scenario is presented below. We will follow Tom as he configures ITA’s
Event Management System to recognize and page/email him on his custom
events.
Tom monitors a room of servers for a Web hosting service. In this room are servers from
Dell, Sunny Computer Corp., and GenericServers.com. Unfortunately, space is at a
premium and the amount of equipment in the room has started to create ventilation
problems, resulting in some systems overheating. Tom wants to make sure that he is
alerted when a system starts to show overheating problems, and he wants to be able to do
this with one application for all of his servers.
August 2001
Page 12
Dell Enterprise Systems Group
Tom determines that microprocessor temperature is a good parameter to use in
determining whether a server is overheating. Fortunately, all three of his server vendors
provide agents that monitor various temperature probes within the server and send traps
based on preset thresholds.
Tom finds that Dell OpenManage IT Assistant can notify him by pager when an SNMP
trap is received. Tom also learns that Dell has pre-populated IT Assistant's event
management database to recognize traps from Dell agents (such as Dell OpenManage
Server Agent). However, before Tom can set up an event filter and event actions based on
that filter, he must configure IT Assistant to recognize the traps from the other two
vendors. Tom can do all of this from the IT Assistant Web-based user interface.
First, Tom needs to look at the trap definitions for the other two server vendors —
unfortunately, he finds that each does things a little differently. Tom is only interested
when a temperature probe goes to a warning or critical status, not if it returns to a
normal status. Tom finds that Sunny Computer Corp. servers produce separate traps for
each severity level change of a temperature probe. In other words, the servers generate a
separate trap (each with a different specific trap ID) for each possible temperature probe
status (normal, warning, and critical). He also finds that the fourth varbind of the trap
contains a text string that specifies the location of the temperature probe.
Tom gathers the data he needs to successfully configure IT Assistant to recognize the
trap: enterprise OID, generic trap ID, specific trap ID, what severity the trap represents,
and the relevant varbind information. He does this for both the warning and critical
temperature probe traps. Tom now needs to configure IT Assistant.
On the IT Assistant left-panel menu, he chooses Event Categories under the
Configuration menu group. Tom could create a new category for the trap event sources
generated by the other two server agents, but he decides to put them under the same event
category and event type used by the pre-populated Dell temperature probe trap. The
event category for this trap is Environmental and contains the event types
Temperature Warning for warning traps and Temperature Failure for critical traps.
Tom starts by setting up the warning trap. First, he chooses the event type Temperature
Warning under the event category Environmental and clicks Edit… Tom needs to add
an event source definition for the trap, so he clicks Add Event Source… and then
SNMP. He names the event source Sunny ComputerCorp. Agent and completes the
generic trap ID, specific trap ID, and enterprise OID. He also chooses a severity level
that the trap represents.
The Format String field contains the text of what Tom will see when he receives this
alert from IT Assistant, so he wants to ensure that it contains meaningful information.
He fills it in with the following text:
A temperature probe warning has been received from system
$n at date $d and time $t: the location of the probe is $4.
Notice that Tom uses available variable substitutions as defined in the ITA User’s Guide.
The $4 represents the fourth varbind of the event source, starting from a count of 1. Tom
clicks OK to complete setup of the temperature warning event source. Then he chooses
the Temperature Failure event type and sets it up the same way as he set up the
August 2001
Page 13
Dell Enterprise Systems Group
temperature warning. When he completes setup of both sources, he confirms his additions
by clicking OK all the way back up to the Event Categories dialog box.
Next, Tom needs to add the event source for his GenericServers.com servers. He needs to
approach this event differently because the status of the temperature probe Tom wants to
monitor is actually in one of the varbinds of the trap. Tom puts this event under the event
category Environmental as he did for Sunny Computer Corp. This time, however, he
decides to create his own event type because the same trap will be received for both
warning and critical temperature change events; therefore, the trap does not fit any single
severity event type (such as Cooling Device Failure or Cooling Device Normal). Tom
selects Environmental, and then clicks Add Type. He names the event type
Temperature Probe Status change. He also gives the event type an optional
description in the Description field. To configure the event source, Tom once again
clicks Add Event Source… then clicks SNMP.
At this point, Tom needs to consult the MIB or trap events file provided by
GenericServers.com for its agent. He finds the following information for the trap he is
interested in: a specific trap ID of 1000, a generic trap ID of 6, and an enterprise OID of
.1.3.6.1.4.1.300.100.1.1.1. Tom also finds out what each varbind sent with the trap
contains. He knows that he needs the two varbinds that provide the severity of the event
and the location of the probe.
NOTE: Each vendor agent's traps are different; these values are not
guaranteed to be in a trap.
Tom knows that each varbind has it own OID assigned to it, which he also knows should
be the enterprise OID followed by additional information to identify that particular
varbind. IT Assistant needs this information for the varbind that contains the severity of
the event to be able to access its value. He has determined this OID to be
.1.3.6.1.4.1.300.100.1.1.1.3. Tom also needs to figure out what values this varbind
contains and what they mean (such as 1 = OK, 2 = warning, 3 = critical) so that he can
map the value to a standard IT Assistant severity level.
From reading the MIB, Tom determines that the GenericServers.com agent produces the
following severity values for the trap and varbind he needs: 1 = other, 2 = unknown, 3 =
OK, 4 = warning, 5 = critical, 6 = non-recoverable. Tom is only interested in values of
warning and critical, but he decides it is best to go ahead and map all the possible values.
Tom also finds that the fifth varbind contains the location of the probe.
Tom is now ready to finish configuring IT Assistant to recognize this trap. Tom enters
the enterprise OID, generic trap ID, and specific trap ID that he noted from the MIB file.
He also enters a similar format string as he did for the Sunny Computer Corp. agent,
noting the differences:
A temperature probe status change has been received from
system $n at date $d and time $t with severity $s: the
location of the probe is $5.
Tom is now ready to configure the severity mapping by value. First, he clicks By Value
on the Severity menu, and then clicks Add Severity. He starts with the severity value of
other: In the Severity combo box, he clicks Unknown (Unknown and Other severities
are represented by the same icon in the IT Assistant Web-based user interface: a gray
August 2001
Page 14
Dell Enterprise Systems Group
question mark). For the Object ID, he enters .1.3.6.1.4.1.300.100.1.1.1.3,
and then for the Object Value he enters 1. He clicks OK, and now the value of 1 is
mapped to Unknown.
Next, he maps Other (or 2) to a severity of Unknown; then he maps OK (or 3) to the IT
Assistant severity of OK, and so on until all six of the values that are possible for the
varbind are mapped to IT Assistant severities. After he defines all severity mappings,
Tom saves all that he has configured by clicking OK all the way back up to the Event
Categories dialog box and then clicks Close on the Event Categories dialog box to
finish his setup.
Tom has just setup ITA’s EMS to recognize his new events from non-Dell agents.
In the following sections, we will see Tom setup his own event filter(s) and
action(s) so that he is paged/emailed should there be any temperature problems.
August 2001
Page 15
Dell Enterprise Systems Group
Section
6
Event Filters and their Configuration in IT
Assistant
Once ITA can recognize an event that comes in, it is able to format the message of
the event into a desirable easy-to-read format. However, the real power of
recognized events comes from setting up event filters and actions. An event filter
in IT Assistant is simply a defined set of conditions as specified by the user. If an
event meets all conditions of the filter, the filter is said to be met for that event –
any event actions linked to that filter will be executed.
Setup of event filters occur from the Filter Configuration dialog, accessible from
the Event Filters link under the Configuration Menu on the ITA Blue Bar. A
configured event filter can have multiple conditions based on event severity, the
time the event is received, the date the event is received, the day of the week the
event is received, what type of event is received, and what node it is received
from, however a minimum set of conditions is required for any event filter: at
least one severity, event type, and event source must be selected. Selecting the
Select All checkbox for the Event Categories / Types dialog section fulfills the
minimum event type requirement and means that any event that is received (and
is defined in the ITA event database) will meet this condition. Selecting the Select
All checkbox for the Select Source Nodes dialog section fulfills the minimum
source node requirement and means that it does not matter where the event
came from – it does not even need to be from a discovered node – for the
condition to be met; see Figure 4.
August 2001
Page 16
Dell Enterprise Systems Group
Figure 4: Filter Configuration Dialog in ITA showing a Minimum Filter
Configuration
For creating custom filters, the following conditions will be applied (if selected):
August 2001

Date Range: If the event is received in the specified date range, the date
range condition is met. The date the event is received is the current date of
the machine that is running the ITA services tier.

Time Range: If the event is received in the specified time range, the time
range condition is met. The time the event is received is the current time of
the machine that is running the ITA services tier.

Days: If the event is received on the day selected, the day of the week
condition is met. The day the event is received is determined by the current
date of the machine that is running the ITA services tier.

Event Categories / Types: Any combination of event types / categories can
be selected. Check the Select All checkbox to remove the event type as a
condition (any event will pass, as long as it is defined in the ITA event
database).
Page 17
Dell Enterprise Systems Group

Source Nodes: Any combination of groups or individual nodes can be
selected. Check the Select All checkbox to remove the source of the event as a
condition (any source that generates the event is allowed). Note that for ITA
v6.1, the “custom groups” group has been removed. Any custom groups
defined will show up on the same level as servers, desktops, etc.
A special note about selecting source nodes:
When a source node is selected, the ip address or hostname contained in the
event (DMI indication or SNMP trap) must match the source node name as it
appears in the source nodes section exactly. In some cases, ITA cannot resolve
the ip address or hostname contained in the event as it was able to during
discovery because discovery will try to resolve the ip address of a node to a
hostname via the remote host’s instrumentation if the domain name service
(DNS) fails; it does not do this for events. However, ITA 6.1 does have enhanced
node name resolution as follows:
ITA 6.0.x resolves an ip address contained in the event via DNS only.
ITA 6.1 resolves an ip address contained in an event as follows:

If DNS is preferred as the name resolution during discovery (default), DNS is
used first to resolve the ip address to a name; if DNS fails, ITA does a
“reverse lookup” in its database of discovered nodes to try to resolve the ip
address to a name; if this fails, the ip address is kept as the source name of
the event.

If instrumentation is preferred as the name resolution during discovery, ITA
first does a reverse lookup in its database of discovered nodes to try to
resolve the ip address to a name; if this fails, then DNS is tried; if this fails,
the ip address is kept as the source name of the event.
For a real-world example in setting-up event filters, we revisit Tom as he
configures ITA for his needs:
Tom has the temperature events that he is interested in configured for the Sunny
Computer Corp. and GenericServers.com agents. Now he needs to set up a filter for his
new events. On the IT Assistant left-panel menu, Tom clicks Event Filters, then clicks
Add... in the Event Filters dialog box.
For Filter Name, he enters Toms Temperature Events - Warning and
Critical only. He then selects Warning and Critical for the severity
configuration. He doesn't care about time filtering, so he does not perform time or date
configuration.
Tom is only interested in temperature changes, so in Select Event Categories / Types
under Environmental, he selects the event types that include the new event sources that
he has entered. For his Dell servers, he also selects all event types that include
Temperature in their names.
In Select Source nodes, Tom selects the servers group because he is only interested in
his servers as the indicator of ventilation problems. Tom clicks OK to accept the filter
setup.
August 2001
Page 18
Dell Enterprise Systems Group
Event filters can be setup to view groups of events in the ITA Alerts window.
Any filter created will be added to a drop-down combo box that is available in
the Alerts window. Selecting a filter will only show those events that the filter
has been met for. This feature is meant to aide the user in categorizing events.
August 2001
Page 19
Dell Enterprise Systems Group
Section
7
Event Actions and their Configuration in IT
Assistant
Event filters are rarely useful without event actions tied to them. There is a welldefined line between the two; filters only evaluate conditions on an event and, if
the filter ultimately evaluates to true, event actions linked to the filter perform
special notifications as configured by the user. These notifications include email,
paging, alert popup, application execution, and, in ITA 6.1, insertion into the NT
event log. Event actions can be setup from two places: the Event Filters dialog
box (actions can only be added here, not edited or deleted), or the Event Actions
link under the Configuration Menu on the ITA Blue Bar, see Figure 5.
To setup an event action, select the Event Actions link on the ITA Blue Bar, then
select New...
There are two boilerplate options from which to create actions: templates and
existing. Template actions are pre-configured actions that ship with ITA. Existing
actions are actions that the user has already setup; the user can pick an already
setup action and derive a new action from it. This feature is handy for those
wishing to create several actions that have minor differences, such as who will be
the recipient of the action. The following actions are available:
August 2001

Alert: This action will popup an alert dialog box on any ITA user interface
that is currently running and connected to the ITA services layer that
receives the event. The alert popup action is pre-configured and cannot be
edited by the user. The message shown is as defined in the format string field
of the matched event. This action is available from the Configure Filter
Actions dialog, which is accessible from selecting the Actions… button in the
Event Filters dialog.

Application Launch: This action will launch any application on the ITA
services tier when the event is received. It is recommended to assure that the
entire path be provided in the executable name field. For the arguments field, it
is possible to perform variable substitutions such as host name, the event
message, date, time, and severity. These variable substitutions are meant to
provide details about the event to the application being launched. Click on
the context-sensitive help button in the Event Action Definition dialog box
for a list of all possible variable substitutions.

Page: This action will page the recipient in the To: field of the Page Action
dialog when the event is received. Before setting-up the page action, a
recipient of the page must be setup in the Winbeep program, which is
Page 20
Dell Enterprise Systems Group
shipped and installed with ITA2. This recipient is known as a subscriber and
it is the name of the subscriber that should be entered in the To: field of the
Page Action dialog. It is important to note that some providers place a limit
on the number of characters that are allowed in a page; if this limit is
exceeded, the page may even fail instead of the message being cut-off. If
some events that should trigger pages are not, this may be the reason why.
Winbeep provides a properties page for the various service providers where
the maximum number of allowed characters can be set. In addition, ITA 6.1
allows this to be configured in its dconfig.ini file3, via the MaxPageMssgLength
attribute under the [EVENT_MANAGEMENT_CONFIG] section. If pages
are not being received in general, this can indicate that Winbeep is not
configured correctly for the service provider being used. In addition, the
modem properties for the Operating System may also need to be adjusted to
allow pages to be sent correctly4. It is recommended that each user setup
within the Winbeep application also be tested by sending a test page to that
user from the Winbeep application.
In the page action dialog’s message field are default variable substitutions
that provide details about the message, such as system name, severity, and
date and time of the event. In ITA 6.1, two new variables are available: the
system’s service tag ($st) and the system’s asset tag ($at). The user can
further customize this message field. When ITA sends a page, it uses the
imcpage.exe program under the covers.

Email: This action will send an email via the SMTP service5 in the To: field of
the Email Action dialog when the event is received. The From: field can
contain a valid email address or a fake one, depending on the setup of the
SMTP server. In the case that a bogus email address is tried and does not
work, it is recommended to setup a separate email account for ITA services.
As with the page action, in the email action dialog’s message field are default
variable substitutions that provide details about the message, such as system
name, severity, and date and time of the event, with service tag and asset tag
variables being added in ITA 6.1. If email messages are not being sent from
the ITA services layer when they should be, it is recommended to check and
make sure SMTP services are installed and running on the ITA services
system. Next, the setup of the SMTP server should be checked 6. When ITA
Winbeep is installed on the ITA services tier. It can be accessed from the Start menu on
the Windows desktop.
2
3
Note that any change to ITA configuration files requires a restart of ITA services.
The ITA users guide and/or readme provide useful hints on how to configure Winbeep
and the OS modem properties correctly.
4
The Windows SMTP service is required on the node running ITA services for the email
action to work. See the ITA users guide for more information.
5
Many firewalls are configured to block SMTP messages from unapproved SMTP servers.
That means the administrator must configure the SMTP service on computer running ITA
services to forward SMTP messages to the corporate approved SMTP server.
6
August 2001
Page 21
Dell Enterprise Systems Group
sends an email, under the covers it executes the vbscript file sendmail.vbs
using the executable cscript.exe. For further troubleshooting, this can be used
directly from the command line. The email action template is added during
installation due to install location dependencies, so if the email action is not
available when selecting create action from template, a database error occurred
during install. It is recommended to reinstall ITA if this occurs.

NT Event Log: This is a new event action, introduced in ITA 6.1. This action
will insert an entry into the NT Event Log (under Application Log) of the
system where ITA services are running. The message shown is as defined in
the format string field of the matched event. Like the alert action, this action is
pre-configured and cannot be edited by the user. This action is available
from the Configure Filter Actions dialog (see Figure 5), which is accessible by
selecting the Actions… button in the Event Filters dialog.
Figure 5: Configure Filter Actions Dialog with Four Actions Configured
Actions are executed in a particular order: as they appear in the Assigned Actions
window of the Configure Filter Actions dialog. Using the Move Up and Move
Down buttons, this order can be changed. The success or failure of an action does
not influence the execution of actions further down the line. In addition, ITA
does not wait for an action to complete before executing the next one.
If the user wishes to delete an event action, he/she may want to view what filters
use that action for alerting. ITA provides an easy way to do this. In the Event
Action dialog, select an action; the Filter Dependencies section below will show
what filters use that action.
August 2001
Page 22
Dell Enterprise Systems Group
For a real-world example in setting-up event actions, we again revisit Tom as he
finishes his configuration of ITA:
Next, Tom sets up actions for his filter. In the Event Filters dialog box, he selects his
new filter and clicks Actions .... First, Tom creates an email action. He selects New
Action .... Tom is creating a new action, not a derivation of one, so he chooses Create
from Template, then EMail, then Create .... Name the action Email Tom. Tom
inserts his email address in the To: field. In the From: field, Tom inserts an email alias
that is meaningful, yet will not be filtered out by his email server. Changing the subject
and message fields is optional, so Tom leaves them as they are. Next, Tom creates a
paging action. Before he does this, Tom runs Winbeep's configuration utility and creates
a subscriber ID for himself, such as "TomsPager." Tom creates a paging action from a
template and inserts the subscriber ID of TomsPager in the To: field of the paging
action he calls Page Tom.. Again, changing the message is optional. Finally, Tom
decides he wants the ITA user interface to display an alert popup, so he selects that
option. He then clicks Close, then Close once again.
Tom will now receive a page, email, and an alert popup when a temperature probe status
goes to a value of warning or critical. He can then look at the location string, which
prints as part of the alert message, to see if it is a microprocessor temperature probe.
August 2001
Page 23
Dell Enterprise Systems Group
Section
8
Event Logging and Configuration in IT
Assistant
Once ITA receives an event, it will store that event in it database, unless it is
thrown away via a discard filter. The discard filter is a single filter that, when all
conditions evaluate to true for an event, that event is discarded as soon as it is
received. Several discard filters can be setup, but only one can be active at any
one time. A typical use for a discard filter is to filter out events that are of no
concern to the typical sys admin, such as SNMP cold start messages. The discard
filter could also be used for events that are emitted every few seconds, but are of
no concern for the moment because the sys admin is already aware of them.
However, ITA also has an event cache that assists in throwing away duplicate
events such as these. Events that come in are checked against the cache, and if an
exact match is found, the event is thrown away.
As mentioned before, ITA does not attempt to match events from different
protocols, so an SNMP trap and DMI indication that are representative of the
same event will be viewed as different. There are two configuration parameters
for the event cache: event cache size and event cache lifetime. The event cache size
refers to the maximum number of events the cache is allowed to hold until it is
flushed; additional events are thrown away until the cache is flushed.
The event cache lifetime refers to the time interval when the cache is flushed
every time. When the cache is flushed, it is possible for the repeating event to be
allowed through again because the cache holds no duplicates. Therefore, this
event, rather than being seen every few seconds, could be seen at every cache
flush interval. The main purpose of the event cache is to prevent the user from
being notified on duplicate events, and to avoid filling-up the event table in the
database.
Another parameter that is configurable in the event logging dialog is the max log
size. This is the maximum number of event entries that are allowed to be stored
in the ITA database. The number of events already in the database is monitored
several times a day, and if the number comes within several hundred events of
hitting the maximum allowed number of events, a message dialog is broadcasted
to every ITA user interface connected to that ITA services tier that warns the user
to delete events or increase the maximum number of allowed events. If the user
chooses to ignore this warning, events received after the event log is full will be
thrown away and not stored.
New to ITA 6.1 is a configurable parameter that allows the user to select when he
wishes to be warned about the event log nearing capacity. Also new is a
August 2001
Page 24
Dell Enterprise Systems Group
configurable parameter that instructs the event management system how many
event log entries to purge when the event log reaches capacity. Only the oldest
events are purged. When ITA 6.1 purges events, a special message describing
how many events were purged is added to the ITA alerts log so that the sys
admin is aware of this. Lastly, in ITA 6.1, the maximum number of allowed
events has been raised significantly to support the management of more nodes,
see Figure 6.
Figure 6: Event Logging Configuration dialog from IT Assistant 6.1
August 2001
Page 25
Dell Enterprise Systems Group
Section
9
Events Generated by IT Assistant
New to ITA 6.1 is the ability to send traps to indicate if an already-discovered
system has just come up or has gone down. These traps are known as system
up/system down traps. When a system goes down suddenly, in most cases there
is no warning from the monitoring agent on that system. Since ITA performs
status polling of its discovered systems, it can provide this notification for the
agent on the managed system. Upon each status poll, if a system that was
previously up has been detected as not responding, ITA will send a system down
trap for that system. If the system was previously down during the last status
poll, but has now been detected as up, a system up trap will be sent for that
system.
ITA does not send “heartbeat” traps – events that constantly indicate the
discovered node’s status upon each status poll. To do so would flood the
network with events upon each ITA status poll. ITA will not send traps unless it
is specifically configured to do so. ITA has its own configuration file to configure
destinations for the traps to be sent to – it does not use the SNMP agent on the
system to do this. In other words, configuring trap destinations for the SNMP
agent on the ITA services tier has no impact on where ITA will send the traps
that it generates. To configure ITA to send system up/system down traps, open
the trapconfig.cfg file in the ITA configuration directory and follow the setup
instructions carefully7. ITA can send traps to multiple destinations, and even to
itself, if desired.
7
August 2001
A restart of ITA services is required for the changes to take affect.
Page 26
Dell Enterprise Systems Group
Section
10
Event Management and the IT Assistant
Database
As mentioned previously in this paper, ITA keeps all information about the
event management system in its database. The organization of information is
kept in such a way as to maximize normalization of the data 8. See Table 3 for
those database tables related to the event management system. To view any of
these tables, as well as perform additional tasks associated with the event
management system, it is recommended to use ITA’s database management
utility dcdbmng.exe, located in ITA’s bin directory.
Table 3: Database tables associated with ITA’s EMS
Table Name
EventCategory
Primary Key(s)
Name
EventType
EventCategoryName,
EventTypeName
EventCategoryName,
EventTypeName,
EventSourceName
EventCategoryName,
EventTypeName,
EventSourceName,
ConfigurationID
EventCategoryName,
EventTypeName,
EventSourceName
Name
SNMPEventSource
SNMPEventSeverityConfigur
ation
DMIEventSource
EventFilter
StoredAction
StoredActionName,
StoredActionID
EventAction
ID
Event
EventID
Purpose of Table
Stores the names of all event
categories.
Stores the names of all event
types and descriptions.
Stores all defined SNMP
event sources.
Stores all defined event
severity information for those
SNMP event sources that
need it.
Stores all defined DMI event
sources.
Stores configured event
filters.
Stores event actions
configured by the user. Also
stores event templates
shipped with ITA.
When an action is linked to a
filter, the link is stored in this
table.
Stores all events received.
Normalization is a database term – the more data is normalized, the less chance there is
of unneeded duplication of the data across multiple tables.
8
August 2001
Page 27
Dell Enterprise Systems Group
The database management utility can assist in the following functions9:

Restore ITA’s EMS to original state – To perform this function, select the
menu item File … Restore Database … Event Management System. Warning –
this will restore the EMS to a state equal to that of a fresh ITA install – all
user customizations will be lost!

Purge all events from the alert/event log at once – To perform this function,
select the event table, and then select the menu item Action … Clear Table. All
events will be deleted.

Export all events from the alert/event log at once – To perform this function,
select the event table, and then select the menu item Action … Export Table.
Note that ITA 6.1 provides additional features in the file … save dialog that is
presented. All events will be exported.

Migrate EMS setup to another ITA installation – In a future release of ITA,
an automated way will allow for the migration of a single EMS
configuration. Until then, the following procedure will produce the desired
results.
If any event categories, event types, or event sources have been customized, then
these settings must be migrated along with the event filters and actions. If they
have not been customized, this step can be skipped. To migrate these, perform
the following functions using the database management utility:
1) On the source machine, select each of the following individual tables and
choose the menu item Action … Export Table to export it. All tables
should be exported to a common folder: EventCategory, EventType,
SNMPEventSource, SNMPEventSeverityConfiguration,
DMIEventSource.
2) On the target machine, select each table mentioned in step 1) above and
clear it by selecting the table, then selecting the menu item Action …
Clear Table.
3) On the target machine, import each file created in step 1) above to its
corresponding table by select that table, then choosing the menu item
Action … Import Table. Make sure to import all 5 files into their
corresponding tables.
4) On the target machine, restart ITA services.
To migrate event filters and event actions to another computer running ITA
services, do the following steps. Note that the email action template is dependent
Because ITA services cache data from the database, it is strongly recommended to restart
services after performing any of the functions mentioned.
9
August 2001
Page 28
Dell Enterprise Systems Group
on where ITA is installed, so it is important that ITA is installed in the same exact
directory structure on the target machine as ITA on the source machine is 10:
1) On the source machine, select each of the following individual tables and
choose the menu item Action … Export Table to export it. All tables should be
exported to a common folder: EventFilter, StoredAction, EventAction.
2) On the target machine, select each table mentioned in step 1) above and clear
it by selecting the table, then selecting the menu item Action … Clear Table.
3) On the target machine, import each file created in step 1) above to its
corresponding table by select that table, then choosing the menu item Action
… Import Table. Make sure to import all 5 files into their corresponding tables
4) On the target machine, restart ITA services.
Note that the database management utility allows the
importing/exporting/clearing of tables from the command line, so a script could
be written to perform the above actions. See the IT Assistant Database Managers
User’s Guide for more information.
When upgrading from ITA 6.0.x to ITA 6.1, the database will be preserved by
default so that no user customizations are lost. This is usually the desired effect,
and the install will only add new events to the database, not modify or delete
events already contained in the database. However, some improvements are
occasionally made to existing events in newer ITA versions, such as
organizational improvements, and this can only be taken advantage of by
receiving a fresh install of the EMS database tables. If a fresh install of the EMS
tables is desired, rather than choosing not to preserve the database or doing an
uninstall before choosing to upgrade to a newer version, it is recommended to
perform a normal upgrade, then use the database management utility to return
the EMS to its original state (see the function Restore ITA’s EMS to original state
above).
An alternative to this is to export the StoredAction table, then edit the directory path to
the sendmail.vbs script in the exported file before it is imported into the target machine,
but this is an advanced configuration left up to the reader’s discretion.
10
August 2001
Page 29
Dell Enterprise Systems Group
Section
11
Conclusion
Starting with version 6.0, OpenManage IT Assistant now provides a powerful
means to event management and alerting. As has been shown, there are many
ways for the user to configure ITA to meet his/her needs in a critical computing
environment where uptime is everything and downtime is money lost. As new
versions of ITA are released, ITA’s event management system will only grow in
functionality and versatility to provide the user a way to monitor his/her entire
computing environment from one central console.
Dell, OpenManage, PowerEdge, PowerVault, and PowerApp are trademarks of Dell Computer Corporation.
Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and
names or their products. Dell disclaims proprietary interest in the marks and names of others.
©Copyright 2001 Dell Computer Corporation. All rights reserved. Reproduction in any manner whatsoever without the
express written permission of Dell Computer Corporation is strictly forbidden. For more information, contact Dell. Dell
cannot be responsible for errors in typography or photography.
Information in this document is subject to change without notice.
August 2001
Page 30
Dell Enterprise Systems Group