IBM Security Host Protection
for Windows Servers
formerly IBM Security Server Protection for Windows
򔻐򗗠򙳰
Administration Guide for Host Protection
for Windows Servers
Version 2.2.2
Copyright statement
© Copyright IBM Corporation 2005, 2012.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.
Contents
Overview . . . . . . . . . . . . . . v
What's new and changed in IBM Security Host
Protection for Windows . . . . . . . . . . v
How to use the documentation . . . . . . . . vi
Chapter 1. Introducing IBM Security
Host Protection for Windows. . . . . . 1
Layered security . . . .
About policy inheritance .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. 1
. 3
Chapter 2. IBM Security Host Protection
for Windows Server agents deployment . 5
Deployment scenarios . . . . . . . . . . . 5
Deployment scenario checklists . . . . . . . 6
Before you deploy an agent . . . . . . . . . 7
Verifying agent licenses . . . . . . . . . . 7
Updating the SiteProtector database . . . . . 8
Updating the Agent Manager . . . . . . . . 9
Setting up SiteProtector Groups . . . . . . 10
Upgrading a version 2.2.x agent . . . . . . . 11
Migrating policy settings from version 1.0 to
version 2.x. . . . . . . . . . . . . . 12
Migrating policy settings from version 2.1 or
2.2.1 . . . . . . . . . . . . . . . . 13
Process overview for upgrading use the heartbeat
mechanism . . . . . . . . . . . . . 14
Task 1: Upgrading the agent version . . . . . 14
Task 2: Upgrading Policies . . . . . . . . 15
Process overview for upgrading using the agent
build mechanism . . . . . . . . . . . 17
Upgrading Policies . . . . . . . . . . . 18
Policy configuration . . . . . . . . . . . 19
Process overview for configuring policies . . . 19
Determining your policy inheritance needs . . . 19
Overriding parent policies . . . . . . . . 20
Configuring policies . . . . . . . . . . 20
Creating and Deploying an Agent Build . . . . . 22
Generating an Agent Build . . . . . . . . 23
Installing an Agent . . . . . . . . . . . 24
Chapter 3. Firewall configuration . . . 25
Key concepts for firewall . . . . .
Protection levels . . . . . . . .
Firewall blocking . . . . . . .
Firewall rules . . . . . . . . .
Firewall precedence . . . . . .
Examples of firewall precedence .
Managing conflicts between firewall
Opening a port through the firewall .
Advanced firewall configuration . .
. .
. .
. .
. .
. .
. .
rules
. .
. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
25
26
27
28
29
29
30
31
32
Chapter 4. Protecting against
intrusions . . . . . . . . . . . . . 33
About security events .
.
.
© Copyright IBM Corp. 2005, 2012
.
.
.
.
.
.
.
. 34
Intrusion prevention or intrusion detection .
Security events . . . . . . . . . .
Descriptions for security event columns .
Trusted addresses . . . . . . . . .
Tuning the agent . . . . . . . . .
Event filtering . . . . . . . . . .
Advanced security event configuration . .
Back tracing . . . . . . . . . . .
Packet logging . . . . . . . . . .
Evidence Logging . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
35
35
36
36
37
39
40
40
41
42
Chapter 5. Protecting against buffer
overflow exploits . . . . . . . . . . 43
Buffer overflow exploit prevention (BOEP) . . . . 43
Understanding buffer overflow exploit prevention
(BOEP) . . . . . . . . . . . . . . . . 44
Chapter 6. Enforcing antivirus
compliance . . . . . . . . . . . . . 47
Understanding antivirus compliance . . . .
Configuring antivirus compliance . . . . .
Advanced antivirus compliance configuration .
.
.
.
. 47
. 48
. 49
Chapter 7. Enforcing application
control . . . . . . . . . . . . . . . 51
Key concepts for application control . . .
How to use application control . . . . .
How to handle unknown applications . .
How to handle known applications . . .
Preparing and importing known applications
Exporting event details from SiteProtector
Editing the file . . . . . . . . .
Importing known applications . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
51
52
53
54
56
57
57
58
Chapter 8. Auditing system integrity
and policy compliance . . . . . . . . 59
Auditing operating system logs. . . . . . .
Auditing Windows event logs . . . . . .
Creating exceptions to audit rules . . . . .
Auditing registry entries . . . . . . . . .
Monitoring the registry . . . . . . . .
Creating exceptions to registry monitoring . .
Coalescing events . . . . . . . . . . .
Identifying a suspicious series of events . . . .
Auditing files and directories . . . . . . .
Monitoring directories and files. . . . . .
File attributes the agent can monitor . . . .
Importing a list of files to monitor . . . . .
Database maintenance schedule . . . . .
Scheduling a baseline comparison . . . . .
Updating the baseline . . . . . . . . .
Excluding directories and files from monitoring
Auditing text logs . . . . . . . . . . .
Text logs . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
59
60
61
61
62
63
63
64
65
65
66
67
67
68
68
69
. 70
. 70
iii
Data identifiers for text log events . . .
Wildcards for specifying log file names .
.
.
.
.
. 71
. 72
Chapter 9. Configuring bypass filters
Bypass filtering .
.
.
.
.
.
.
.
.
.
.
73
.
. 73
Chapter 10. Configuring administrative
settings . . . . . . . . . . . . . . 75
Configuring management settings . . . . . .
Enabling agent protection . . . . . . . .
Protecting during system start-up and shutdown .
Setting up configuration sharing . . . . . .
Configuring event caching . . . . . . . .
.
.
.
.
.
75
77
78
79
80
Not seeing any BOEP alerts . . . . . . . . .
Is Version 2.0 supported on ISA Server? . . . . .
How can I restore custom policy settings for a child
group? . . . . . . . . . . . . . . . .
Overriding parent policies does not give child
policies with the parent policy . . . . . . . .
Under heavy traffic conditions traffic seems to be
bypassing analysis . . . . . . . . . . . .
Not seeing any file integrity monitoring alerts . . .
Traffic seems to be bypassing analysis . . . . .
Refresh agent feature in the SiteProtector System not
functioning . . . . . . . . . . . . . .
Agent showing as offline . . . . . . . . . .
System tray icon disappeared after upgrade . . .
83
83
84
84
85
85
86
87
87
88
Chapter 11. Troubleshooting and
support . . . . . . . . . . . . . . 81
Notices . . . . . . . . . . . . . . 89
Contacting IBM Support . . . . . . .
Agent build package cannot be installed due
missing policies . . . . . . . . . .
Reporting a false positive. . . . . . .
Index . . . . . . . . . . . . . . . 91
iv
. .
to
. .
. .
Trademarks .
.
.
.
.
.
.
.
.
.
.
.
.
. 90
. 81
. 82
. 82
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Overview
This guide is designed to help you use the IBM Security Host Protection agent to protect servers and
ensure that they are in compliance with your corporate policy.
Scope
This guide describes the features of IBM Security Host Protection for Windows agents and explains how
to configure policies from SiteProtector™ and deploy agents.
This guide has been updated for IBM Security Host Protection version 2.2.2.
Audience
This guide is intended for security managers who manage IBM Security Host Protection for Windows
agents from SiteProtector.
What's new and changed in IBM Security Host Protection for Windows
This topic describes the new and changed features that are in the IBM Security Host Protection for
Windows release.
What's new in IBM Security Host Protection V2.2.2
The product formerly known as IBM Security Server Protection for Windows/Proventia Desktop is
renamed to IBM Security Host Protection and the user interface was changed to reflect the new name.
This release includes the following new features and enhancements:
v IPv6 support
IBM Security Host Protection for Windows Version 2.2.2 supports IPv6 addresses for all policies that
support IPv4 addresses including Firewall Rules, Bypass Filters, Security Events, and Group Settings
for Windows Server 2008 and later.
v New default installation folder in update settings policy
The default installation folder was changed from %ProgramFiles%\IBM\Server Protection to
%ProgramFiles%\IBM\Host Protection.
v Text log monitoring
The text log monitoring component in the IBM Security Host Protection for Windows agent monitors
Windows Server activity in the specified log files so that you can identify suspicious activity to your
system.
v Improvements to file integrity monitoring
The file integrity monitoring component in the IBM Security Host Protection for Windows agent now
includes the ability to perform maintenance on the database to reduce unused storage.
v Changes to system integrity monitoring and registry integrity monitoring
The registry integrity monitoring component is now named Registry Event Rules and is located in the
system integrity monitoring policy in the IBM Security Host Protection for Windows agent.
v Log file rollover
The firewall log file (blackd.log) rolls over to maintain the previous five versions. The old versions are
named 1_blackd.log, 2_blackd.log and so on.
v Agent version 2.2
© Copyright IBM Corp. 2005, 2012
v
All references to agent version 2.2 in this guide are now referred to as version 2.2.1
How to use the documentation
Use this documentation to help you set policies for IBM Security Host Protection for Windows agents.
License agreement
For licensing information on IBM® Security Solutions products, download the IBM Licensing Agreement
from: http://www.ibm.com/services/us/iss/html/contracts_landing.html
vi
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 1. Introducing IBM Security Host Protection for
Windows
This chapter introduces the IBM Security Host Protection for Windows agent and describes how the agent
uses a layered security approach to protect your computer. This chapter includes a process overview for
using SiteProtector System to prepare, create, deploy, and manage agents.
Topics
“Layered security”
“About policy inheritance” on page 3
Layered security
IBM Security Host Protection for Windows provides multiple lines of defense by protecting the network
vector, the application vector, and by monitoring files to detect unauthorized changes to the system. This
layered protection approach provides greater overall protection for your host.
Protection from network threats
Network-based attacks use malicious network traffic to remotely compromise target systems. Unlike other
threats, network-based attacks can penetrate a system, run, and propagate without human intervention.
Network attack types include automated and direct attacks executed through worms, denial of service
(DoS) attacks, and the installation of remote access backdoors.
IBM Security Host Protection for Windows provides the following components to prevent network-based
attacks:
v Firewall (FW)
The firewall is the first line of defense against a network-based attack. The firewall can block inbound
and outbound packets from particular IPv4 and IPv6 addresses, port numbers, or protocols. By limiting
inbound and outbound traffic to a set of predefined services and participants, organizations can
minimize their exposure to threats.
v Intrusion Prevention
After authorized packets pass through the firewall the agent analyzes them for malicious content. The
agent drops offending packets, and allows the original traffic to continue through the network stack
unhindered. If the agent detects malicious content that poses an immediate threat to the system, it can
block the packet and reset the connection. The agent can also configure the firewall to block all traffic
to or from the intruder.
This layer of defense is particularly effective because it works at the network-level. Inbound packets
are immediately inspected before they can traverse to vulnerable sections of the operating system. As a
result, this protection provides the equivalent of in-line inspection.
v Buffer Overflow Exploit Prevention (BOEP)
The buffer overflow exploit prevention (BOEP) component is the last layer of preventative defense. The
BOEP component monitors for attacks that overrun writable memory regions or that attempt to access
unauthorized execution spaces.
Note: Note: Buffer Overflow Exploit Prevention is not currently supported on any 64-bit system,
Windows Server 2008, or Windows Server 2008 R2.
© Copyright IBM Corp. 2005, 2012
1
Protection from application threats
Application-based attacks use program files (or attachments) to attack and compromise target systems.
Unlike network-based attacks, program files typically require some form of user involvement to initiate
an attack. Email, files intentionally and unintentionally downloaded from the Internet, injected Web sites,
popup windows, removable media, and peer-to-peer applications are common sources of
application-based attacks.
IBM Security Host Protection for Windows provides the following components to protect against
application-based attacks:
v Antivirus (AV) Compliance
Antivirus software is the computer's first layer of defense against an application-based attack.
Signature-based antivirus products are extremely effective in detecting and preventing known viruses,
worms, and some Trojans.
The IBM Security Host Protection for Windows agent can monitor the host system for the presence of
antivirus software and required updates. You can block network access until the antivirus software has
been updated.
v Application Control (AC)
Application control is the last layer of preventative defense from an application-based attack. When an
unknown application tries to start, the agent can terminate the application or allow it to run. When an
unauthorized application tries to connect to a network, the agent can block its network access,
terminate it, or allow it to connect.
Note: Application control is not currently supported on 64-bit systems or Windows Server 2008.
Auditing and compliance protection
Audit-based protection detects and alerts on unauthorized changes that can indicate that a host has been
compromised. Unauthorized changes represent one of the largest threats to the security of a system.
Auditing system integrity is a powerful layer of defence for your infrastructure.
IBM Security Host Protection for Windows provides the following components to protect against
unauthorized changes to files:
v File Integrity Monitoring
The file integrity monitoring component automates the detection of unplanned and unauthorized
changes to files so that you can identify and troubleshoot changes that may pose a threat to the system.
v Registry Integrity Monitoring
The Windows registry contains highly sensitive data, such as user profiles and the ports used by the
system, so it is essential that you secure the registry. You can provide a certain level of security by
setting the appropriate access rights to subkeys and registry entries; however, you cannot completely
lock down the registry because users and programs need access to certain areas to function correctly.
Because you must allow some access to the registry, it is essential to monitor activity in the registry so
that you can quickly identify potential threats to your system.
v System Integrity Monitoring
System event logs can hold important clues to problems you may be having with your computer or the
applications that run on your computer. Auditing the event logs can help you identify, diagnose, and
minimize potential threats to system integrity.
v Text Log Monitoring
Text logs contain useful information for analyzing data. Monitoring specified text logs for string
patterns can alert you automatically to important events.
2
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
About policy inheritance
Policy inheritance provides an efficient way to use policies based on parent and child relationships. Child
groups or child agents can inherit policies from parent groups. You can also override inheritance from a
parent by defining a policy at the child level. An agent or group uses the policy defined at its own level.
If there is no policy defined at this level, the policy defined at the nearest parent node where a policy is
defined.
Important: Policy inheritance is a powerful way to manage agent policies. The effects of inheritance can
be far reaching. Review the policy inheritance information in the SiteProtector System Policies and Responses
Configuration Guide before you configure your policies.
Example
You configure the policies for a group of assets called “Mail Servers”. There are 10 subgroups in the Mail
Servers group. If all of the subgroups use the same policy settings, the policy settings can pass from the
Mail Servers group to all 10 subgroups. If certain subgroups in the Mail Servers group require
customized policy settings, you can define a customized policy at the subgroup level for that policy. This
will not affect the policy settings for the other subgroups. This approach is much quicker than
configuring the policy settings for all 10 subgroups.
Policies that are displayed in the left pane
If a policy is defined for a particular group or agent, this policy is listed as a member of the group or
agent when you expand the group or agent in the navigation pane. When you select a policy in the
navigation pane, the contents for that policy is displayed in the right pane.
Policies that are displayed in the right pane
When you select a group or agent in the tree, a list of policies that are in use for the selected object opens
in the right pane.
Overriding a policy for a group or agent
If you need to customize a policy for an agent in a group or for a group in a Site, you can define a
different policy (override the current policy) at whichever level you need.
Reference: See the SiteProtector System Policies and Responses Configuration Guide.
When are policy updates applied to the agent?
When you edit a policy, you must save the changes. It might, however, take some time before you see a
change in agent behavior for the following reasons:
v The agent only periodically checks for policy updates (based on the heartbeat interval)
Note: You can force an agent to apply an updated policy immediately by using the SiteProtector
System Refresh Agent feature.
v Large groups of agents take longer to update
Chapter 1. Introducing IBM Security Host Protection for Windows
3
4
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 2. IBM Security Host Protection for Windows Server
agents deployment
Deploying IBM Security Host Protection for Windows agents is a multi-stage process. This process
includes updating SiteProtector components, configuring policies, and deploying configured policies to
the host system.
Pilot program
Consider running a pilot program before you deploy IBM Security Host Protection for Windows agents in
your production environment. A pilot program is a small scale deployment of agents (usually on
non-production systems) that you can set up to test different policy settings in relative safety. During the
pilot, you can collect information and use it to modify your policies to minimize usability problems when
you deploy agents in a production environment.
Deployment scenario checklists
See the “Deployment scenario checklists” on page 6 to determine how best to use the information.
Topics
“Deployment scenarios”
“Before you deploy an agent” on page 7
“Upgrading a version 2.2.x agent” on page 11
“Policy configuration” on page 19
“Creating and Deploying an Agent Build” on page 22
Deployment scenarios
This section includes deployment scenario checklists to guide you through the deployment process.
Topics
“Deployment scenario checklists” on page 6
© Copyright IBM Corp. 2005, 2012
5
Deployment scenario checklists
Use the appropriate deployment scenario checklist in this topic to guide you through the deployment
process.
Installing an agent for the first time
Use the following checklist to help you deploy an agent for the first time:
U
Complete the tasks outlined in...
h
“Before you deploy an agent” on page 7
h
“Policy configuration” on page 19
h
“Creating and Deploying an Agent Build” on page 22
Updating an earlier 2.x or 2.2.x agent to a later agent version using the heartbeat
mechanism
U
Complete the tasks outlined in...
h
“Updating the SiteProtector database” on page 8
h
“Updating the Agent Manager” on page 9
h
“Configuring policies” on page 20
Updating an earlier 2.x or 2.2.x agent to a later agent version using the agent build
mechanism
U
Complete the tasks outlined in...
h
“Updating the SiteProtector database” on page 8
h
“Updating the Agent Manager” on page 9
h
“Configuring policies” on page 20
h
“Creating and Deploying an Agent Build” on page 22
Updating from a 2.1 or 2.2.1 agent using the heartbeat mechanism
U
Complete the tasks outlined in...
h
“Updating the SiteProtector database” on page 8
h
“Updating the Agent Manager” on page 9
h
“Migrating policy settings from version 2.1 or 2.2.1” on page 13
h
“Process overview for upgrading use the heartbeat mechanism” on page 14
h
“Configuring policies” on page 20
Updating from a 2.1 or 2.2.1 agent using the agent build mechanism
U
Complete the tasks outlined in...
h
“Updating the SiteProtector database” on page 8
h
“Updating the Agent Manager” on page 9
h
“Migrating policy settings from version 2.1 or 2.2.1” on page 13
“Process overview for upgrading using the agent build mechanism” on page 17
6
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
U
Complete the tasks outlined in...
h
“Configuring policies” on page 20
h
“Creating and Deploying an Agent Build” on page 22
Before you deploy an agent
This section outlines the tasks you must complete before you can install or upgrade an agent on your
computer.
Topics
“Verifying agent licenses”
“Updating the SiteProtector database” on page 8
“Updating the Agent Manager” on page 9
“Setting up SiteProtector Groups” on page 10
Verifying agent licenses
About this task
You must have valid licenses before you can install IBM Security Host Protection for Windows agents.
Verifying Agent Licenses
Procedure
1. In SiteProtector, click Tools > Licenses > Agent/Module. The Agent/Module License Information for
Site window is displayed.
2. Select the Licenses tab.
3. In the Agent Type column, locate the appropriate row, and then verify the following columns in that
row:
v License Limit: Displays the number of available licenses
v State: Displays the status of the license; if the status is Key Good, the license is valid.
Note: If there is any other status, see the SiteProtector Help.
Chapter 2. IBM Security Host Protection for Windows Server agents deployment
7
Obtaining a license file
Before you begin
Ensure that you have a valid license key for your agents.
Adding agent licenses
Procedure
1. In the navigation pane, select the Site Node.
2. Select Tools > Licenses > Agent/Module. The Agent/Module License Information for Site window
opens.
3. Select the Licenses tab, and then click Add.
4. Navigate to the folder that contains the license you want to add, select the license file, and then click
OK.
Note: License key files have either a .key or .isslicense file extension.
The End User License Agreement opens.
5. Click Accept. The license is displayed in the License tab.
Updating the SiteProtector database
IBM Security Systems delivers the policies for IBM Security Host Protection for Windows agents through
a database update package. You must update the SiteProtector system database before you can deploy
IBM Security Host Protection for Windows, Version 2.x and 2.2.x agents.
About this task
Verifying and updating the database involves the following tasks:
U
Task
Reference
h
Verify that the SiteProtector system has
downloaded the database service pack.
Task 1: Verifying the status of the database
h
Apply the database service pack.
Task 2: Applying the database service pack
8
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Task 1: Verifying the status of the database
Procedure
1. In the SiteProtector system, select the applicable group, and then select the Agent view.
2. In the Update Status column, verify the status of the database as follows:
v If the status is Out of Date, there is an update available. Go to “Task 2: Applying the database
service pack” to continue.
v If the status is Current, the update has already been applied. Go to “Updating the Agent Manager.”
Task 2: Applying the database service pack
Procedure
1. In the SiteProtector system, select the applicable group, and then select the Agent view.
2. Right-click SiteProtector Database, and then click Updates > Apply XPU.
Important: After the SiteProtector Database update is complete, close and then reopen the
SiteProtector console. If you do not close and reopen the SiteProtector console, all of your policies
might be missing from the agent build.
Updating the Agent Manager
IBM Security Solutions delivers the software for IBM Security Host Protection for Windows agents
through an Agent Manager update package. You must update the Agent Manager before you can deploy
IBM Security Host Protection for Windows agents.
About this task
Verifying and updating the Agent Manager XPU involves the following tasks:
U
Task
Reference
h
Verify that the SiteProtector system has
downloaded the Agent Manager XPU.
Task 1: Verifying the status of the Agent
Manager
h
Apply the Agent Manager XPU.
Task 2: Applying the Agent Manager XPU
Chapter 2. IBM Security Host Protection for Windows Server agents deployment
9
Task 1: Verifying the status of the Agent Manager
Procedure
1. In the SiteProtector system, select the applicable group, and then select the Agent view.
2. Verify the status of the Agent Manager in the Update Status column as follows:
v If the status is Out of Date, there is an update available. Go to “Task 2: Applying the Agent
Manager XPU” to continue.
v If the status is Current, the update has already been applied. Go to “Migrating policy settings from
version 2.1 or 2.2.1” on page 13.
Task 2: Applying the Agent Manager XPU
Procedure
1. In the SiteProtector system, select the applicable group, and then select the Agent view.
2. Right-click Agent Manager, and then select Updates > Apply XPU.
Setting up SiteProtector Groups
About this task
A group is a collection of network assets and the SiteProtector components or agents that reside on those
assets. A well designed group structure can help you manage your security more efficiently.
Note: If you have already created your group structure within SiteProtector, refer to your deployment
scenario checklist on page 18 for your next step.
Groups provide the following:
v A means for organizing, accessing, and managing important information about the network assets
monitored by SiteProtector and other IBM Security products.
v A means for organizing and managing the other IBM Security products that work with SiteProtector.
v A means for performing SiteProtector tasks on groups of assets and agents, such as applying policies to
agents in a group or viewing the security events for a specific group of assets.
v A navigational tool in the Console that you can use to move between different groups of network
assets and agents as you perform your security management tasks.
Note: If you have already created your group structure within SiteProtector, go to “Process overview for
configuring policies” on page 19.
10
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Adding a new group
Procedure
1. In the navigation pane, do one of the following:
v to add the first group in a site—right-click the Site group, and then select New > Group.
v to add a group to an existing group—right-click a group, and then select New > Group.
The New Group folder appears below the selected group.
2. Type the group name in the highlighted box, and then press ENTER.
Reference: For additional information about setting up groups in SiteProtector, see the SiteProtector
Configuration Guide.
The group appears in the navigation pane.
What to do next
Refer to your deployment scenario checklist in “Deployment scenario checklists” on page 6.
Upgrading a version 2.2.x agent
If you already have Version 2.2.1 agents deployed, you can upgrade these agents to Version 2.2.2. Note
that all instances of version 2.2 are shown as 2.2.1. Upgrading your agents to the latest version allows
you to use the new features to protect your computer.
Prerequisites
See “Deployment scenarios” on page 5.
See “Before you deploy an agent” on page 7.
Upgrading methods
You can upgrade IBM Security Host Protection for Windows agents in the following ways:
v Heartbeat mechanism
v Agent build mechanism
Choosing the right method
Choose the method suitable to you based on the following:
v For systems that are difficult to access, use the heartbeat mechanism
v For all other systems, use either mechanism
Topics
“Migrating policy settings from version 2.1 or 2.2.1” on page 13
“Process overview for upgrading use the heartbeat mechanism” on page 14
“Task 1: Upgrading the agent version” on page 14
“Task 2: Upgrading Policies” on page 15
“Process overview for upgrading using the agent build mechanism” on page 17
“Upgrading Policies” on page 18
Chapter 2. IBM Security Host Protection for Windows Server agents deployment
11
Migrating policy settings from version 1.0 to version 2.x
Before you begin
See “Before you deploy an agent” on page 7.
About this task
Reference: See “About policy inheritance” on page 3 for a high-level overview of policy inheritance; see
the SiteProtector Policies and Responses Configuration Guide for a more in-depth discussion on policy
inheritance.
Settings configured from the local console are not migrated as part of this upgrade. If you want an agent
to continue using settings set from the local console after this upgrade, you must record the local settings
before you upgrade the agent. After the upgrade, you must reapply the settings from the local console.
Procedure
1. In SiteProtector, select the Agent view.
2. In the navigation pane, right-click the applicable group, and then select Updates > Migrate Agent
Version.
The Migrate Agent Version window appears.
3. Click the Details icon, and then select Proventia Server for Windows from the Agent Type list.
4.
5.
6.
7.
8.
12
Click the Upgrade Details icon.
In the Migrate From Agent Version list, select 1.0.
In the Update To Agent Version list, select a 2.x version.
Click OK.
Do one of the following:
v To complete the upgrade using the heartbeat mechanism, go to “Process overview for upgrading
use the heartbeat mechanism” on page 14
v To complete the upgrade using the Agent Build mechanism, go to “Process overview for upgrading
using the agent build mechanism” on page 17
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Migrating policy settings from version 2.1 or 2.2.1
Before you begin
See “Before you deploy an agent” on page 7.
About this task
Important: If you are upgrading your agents to version 2.2.1 or version 2.2.2, you must upgrade the
agent versions before upgrading from Windows 2008 SP2 to Windows 2008 R2.
Settings configured from the local console are not migrated as part of this upgrade. If you want an agent
to continue using settings set from the local console after this upgrade, you must record the local settings
before you upgrade the agent. After the upgrade, you must reapply the settings from the local console.
Procedure
1. In the SiteProtector System, select the Agent view. For SiteProtector 2.9, select the Policy view.
2. In the navigation pane, right-click the applicable group, and then select Updates > Migrate Agent
Version.
The Migrate Agent Version window appears.
3. Click the Details icon, and then select IBM Security Host Protection for Windows from the Agent
Type list.
4. Click the Upgrade Details icon.
5. In the Migrate From Agent Version list, select 2.1 or 2.2.1.
6. In the Update To Agent Version list, select 2.2.1 or 2.2.2.
7. Click OK.
8. Do one of the following:
v To complete the upgrade using the heartbeat mechanism, go to “Process overview for upgrading
use the heartbeat mechanism” on page 14
v To complete the upgrade using the Agent Build mechanism, go to “Process overview for upgrading
using the agent build mechanism” on page 17
Chapter 2. IBM Security Host Protection for Windows Server agents deployment
13
Process overview for upgrading use the heartbeat mechanism
This topic provides an overview of the tasks involved in using the heartbeat mechanism to upgrade
agents from Version 2.1 or Version 2.2.1.
Prerequisites
See “Before you deploy an agent” on page 7.
See “Migrating policy settings from version 2.1 or 2.2.1” on page 13.
Process overview
Upgrading IBM Security Host Protection for Windows agents to version 2.2.1 or version 2.2.2 using the
heartbeat mechanism involves the following tasks:
Task
Description
Reference
1
Upgrade the agent version
“Task 1: Upgrading the agent version”
2
Upgrade policies
“Task 2: Upgrading Policies” on page 15
Task 1: Upgrading the agent version
You must define the new server protection version to upgrade agents. The server protection version
defines the version number of the agents and allows you to manage more than one version of agents
from the same system.
About this task
The following table outlines the process for upgrading the server protection version:
h
Task
Reference
h
Update the server protection version
Task 1: Updating the server protection
version
h
Verify the agent version
Task 2: Verifying the agent version
14
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Task 1: Updating the host protection version
Procedure
1.
2.
3.
4.
5.
In the SiteProtector System, select the applicable group, and then select the Policy view.
In the Agent Type list, select Host Protection for Windows.
If you are migrating from version 2.1, in the Agent Version list, select 2.1
If you are migrating from version 2.2.1, in the Agent Version list, select 2.2.1
In the Policy pane, right-click Update Settings and then select Open or New Policy.
6. If you chose to create a new policy in the previous step, name the new policy and then click OK.
7. For 2.1 agents migrating to version 2.2.1, in the Host protection version list, select the version that
starts with 2.2.1.
8. For 2.2.1 agents migrating to version 2.2.2, in the Host protection version list, select the version that
starts with 2.2.2.
9. In the Install path box, type the path to where the agents in this group are installed.
10. Save the policy.
11. Do one of the following to apply the updated policy:
v Wait for the agents to send a heartbeat to the SiteProtector System
v Select the applicable group, and then select Action > Refresh Agent to force the agents to send a
heartbeat to the SiteProtector System immediately
Task 2: Verifying the agent version:
Procedure
1. In the SiteProtector System, select the applicable group, and then select the Agent view.
2. In the Version column, verify the agent version.
Task 2: Upgrading Policies
About this task
Policy settings determine the behavior of agents. As part of the upgrade you must, at a minimum,
configure the Install and Update Settings policy. This policy defines the Agent Version setting which
defines both the version of the agent software and the build version of that software. This policy also
defines the installation directory for the agent on the host system.
Tip: You can also configure the settings for any other policies. You can edit policies at the parent group
level (settings are inherited by any child groups), or you can override the parent group policies and edit
policies at the child-group level (settings are specific to the child group only).
Chapter 2. IBM Security Host Protection for Windows Server agents deployment
15
Editing policies
Procedure
1.
2.
3.
4.
5.
In the SiteProtector System, select the Agent view.
In the navigation pane, right-click the applicable group, and then select Manage Policy.
In the Agent Type list, select Host Protection for Windows.
For version 2.1 agents, in the Agent Version list, select 2.1.
For version 2.2.1 agents, in the Agent Version list, select 2.2.1.
6. For version 2.2.2 agents, in the Agent Version list, select 2.2.2.
7. In the navigation pane, select the group.
The Active Deployments window for the group appears on the right pane.
8. In the right pane, right-click Update Settings, and select Open.
9. In the Agent Version list, select the agent version.
10. In the Install path box, type the path to where the agents in this group are installed.
11. Click the Save icon.
12. From the Save Policy Version window, enter a Comment and then select the Deploy This New
Version check box.
13. Click OK. The Deploy Policy window opens.
14. Click Targets.
15. Select the group you want to deploy this policy to.
16. Click OK and then close the tab.
17. To edit additional policies, do the following:
v In the left pane, re-select the applicable group
v In the right pane, right-click the policy to edit, and then select Open
v Edit the policy
Reference: See the Help for guidance.
18. Click the Save icon.
19. Repeat Step 12 through Step 18 for each policy you want to edit.
20. Do one of the following to apply the updated policy:
v Wait for the agents to send a heartbeat to the SiteProtector System
v Select the applicable group, and then select Action > Refresh Agent to force the agents to send a
heartbeat to the SiteProtector System immediately
What to do next
v If you are performing the steps to upgrade from version 1.0 to version 2.0, you are finished.
v If you are performing the steps to upgrade from version 2.0 to version 2.1 or version 2.2, you must
complete the procedure for “Migrating policy settings from version 2.1 or 2.2.1” on page 13.
16
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
System tray icon
After you upgrade agents using the heartbeat mechanism, the system tray icon is not visible on the host
system. If you open the console from the Start menu, the icon will again be visible in the system tray.
This only happens immediately following the update to 2.0.
Process overview for upgrading using the agent build mechanism
This topic provides an overview of the tasks involved in using the agent build mechanism to upgrade
agents from Version 1.0, Version 2.0, or Version 2.1.
Prerequisites
See “Before you deploy an agent” on page 7.
See “Migrating policy settings from version 1.0 to version 2.x” on page 12.
See “Migrating policy settings from version 2.1 or 2.2.1” on page 13.
What is an agent build?
An agent build is a pre-configured installation package that includes the following software and
user-defined information:
v The actual IBM Security Host Protection for Windows software that the build installs on the computer
v Information about how the agent should communicate with the SiteProtector System, including the IP
address of the Agent Manager it reports to and the account name and password it should use when it
contacts the Agent Manager
v Policies that define the protection provided by the agent
Process overview
The following table provides an overview of the stages involved in upgrading agents using the agent
build mechanism:
Task
Description
Reference
1
Upgrade policies
Upgrading Policies
2
Generate an Agent Build
Generating an Agent Build
3
Install agents using the Agent Build
Installing an agent
Chapter 2. IBM Security Host Protection for Windows Server agents deployment
17
Upgrading Policies
About this task
Policy settings determine the behavior of agents. As part of the upgrade you must, at a minimum,
configure the Install and Update Settings policy. This policy defines the Agent Version setting which
defines both the version of the agent and the version of the agent build. This policy also defines the
installation directory for the agent on the host system.
Tip: You can also configure the settings for any other policies. You can edit policies at the parent group
level (settings are inherited by any child groups), or you can override the parent group policies and edit
policies at the child-group level (settings are specific to the child group only).
Editing parent level policies
Procedure
1. In SiteProtector, select the Agent view.
2. In the navigation pane, right-click the applicable group, and then select Manage Policy.
3. In the Agent Type list, select Server Protection for Windows.
4. For version 2.1 agents, in the Agent Version list, select 2.1.
5. For version 2.2.1 agents, in the Agent Version list, select 2.2.1.
6. For version 2.2.2 agents, in the Agent Version list, select 2.2.2.
7. In the navigation pane, select the group.
The Active Deployments window for the group appears on the right pane.
8. In the right pane, right-click Update Settings, and then select Open.
9. In the Agent Version list, select the agent version.
10. In the Install path box, type the path to where the agents in this group should be installed.
11. Click the Save icon.
12. From the Save Policy Version window, enter a Comment and then select the Deploy This New
Version check box.
13. Click OK. The Deploy Policy window opens.
14. Click Targets.
15. Select the group you want to deploy this policy to.
16. Click OK and then close the tab.
17. To edit additional policies, do the following:
v In the left pane, re-select the applicable group
v In the right pane, right-click the policy to edit, and then select Open
v Edit the policy
Reference: See the Help for guidance.
18. Click the Save icon.
19. Repeat Step 12 through Step 18 for each policy you want to edit.
What to do next
See “Creating and Deploying an Agent Build” on page 22
18
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Policy configuration
Policies define how the agent protects your computer. By configuring policies and deploying them to an
agent, you determine the level of protection the agent provides.
Topics
“Process overview for configuring policies”
“Determining your policy inheritance needs”
“Overriding parent policies” on page 20
“Configuring policies” on page 20
Process overview for configuring policies
Configure agent policies so that the agent can protect your system as soon as the policies are deployed.
Process overview
The following table outlines the tasks to perform to configure policies:
Task
Description
Reference
1
Determine your policy inheritance needs
“Determining your policy inheritance needs”
2
Implement your policy structure
“Overriding parent policies” on page 20
3
Configure the policies
“Configuring policies” on page 20
Determining your policy inheritance needs
Policy inheritance provides an efficient way to use policies based on parent and child relationships. Child
groups, by default, inherit policies from the nearest parent group. Child groups can, however, override
the parent policies they inherit so that you have more granular control of settings. Policy settings in one
child group do not affect policy settings in the parent group or policy settings in any sibling groups.
Use policy inheritance to implement the policy structure that best meets your security needs. Agents can
use the following policy structures:
v All policies are inherited from a parent group
v All policies override the parent group policies
v A combination of inherited and overridden policies
Reference: For more information about policy inheritance, see the SiteProtector Configuration Guide.
When to override parent policies
Override parent policies at the child level when you need a custom policy setting at the child level.
Considerations
To maximize efficiency, you should configure the parent policy, override the parent policy from the child,
and then customize the child policy. To make configuration changes easier to implement, inherit as many
policies as possible from the parent group. At a minimum, consider inheriting policies that define the
software version and installation path information for the agent; you may want to override policies that
define protection settings so you can have customized protection for child groups.
Chapter 2. IBM Security Host Protection for Windows Server agents deployment
19
Overriding parent policies
About this task
If you determine that you need custom policy settings at the child level, you must override the applicable
parent level policy. You cannot implement policy customization at the child level until you override the
parent level policy.
Tip: Inherit as many policies as possible from the parent group as this makes configuration changes
easier to implement. Consider inheriting at least the Update Settings policy as this ensures you are
running the same version of the agent on all systems and that the agent is installed in the same directory
on each system.
Procedure
1. In the navigation pane, right-click a group and select Manage Policy.
2. In the Agent Type list, select Host Protection for Windows.
3. For agent version 2.1, in the Agent Version list, select 2.1.
4. For agent version 2.2.1, in the Agent Version list, select 2.2.1.
5. For agent version 2.2.2, in the Agent Version list, select 2.2.2.
6. In the navigation pane, select the child group you want custom policies for.
Note: The Active Deployments reflects the current policies assigned to this child group.
The Active Deployments window for the group appears in the right pane.
7. In the right pane, select the policy that you want to override.
Tip: Use the Inheriting From column to determine which parent policies the child currently inherits.
8. Right-click the selected policies, and then click Deploy.
9. Click OK.
The selected policy appears under the child group folder in the navigation pane.
Configuring policies
After you have implemented your policy inheritance structure—that is, you have decided which policies
to inherit from the parent level and which policies to override at the child level—you must configure the
policies so the agent can protect your system.
About this task
You must save and deploy each policy after you have finished configuring it.
Important: While you can use the following procedure to guide you through the configuration of both
parent level policies and child level policies, be aware that changes to parent level policies affect all child
groups.
Procedure
1. In the navigation pane, right-click a group and select Manage Policy.
2. In the Agent Type list, select Server Protection for Windows.
3. For version 2.0 agents, in the Agent Version list, select 2.0.
4.
5.
6.
7.
20
For version 2.1 agents, in the Agent Version list, select 2.1.
For version 2.2 agents, in the Agent Version list, select 2.2.
In the navigation pane, expand the group you are configuring policies for.
If you are configuring policies for a parent group, select Default Repository.
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Important: If the group you are configuring does not contain an Update Settings policy (either by
inheriting or by having a custom version), you must create this policy before you can deploy your
agent. See Update Settings in the table below for more information.
The default policy types appear in the right pane.
8. Configure the appropriate policies based on the following information:
Tip: As you configure each policy, press F1 if you need Help.
For this policy...
do this...
Administration
see Chapter 10, “Configuring administrative settings,” on page 75
Application Compliance
see Chapter 6, “Enforcing antivirus compliance,” on page 47 and Chapter 7,
“Enforcing application control,” on page 51
BOEP
see Chapter 5, “Protecting against buffer overflow exploits,” on page 43
Bypass Filters
see Chapter 9, “Configuring bypass filters,” on page 73
File Integrity Monitoring
see “Auditing files and directories” on page 65
Firewall
see Chapter 3, “Firewall configuration,” on page 25
Group Settings
1. Select the Agent Manager List tab.
2. If the Agent Manager for this group is not in the list, click Add.
3. Click Choose an Agent Manager, select the Agent Manager for this group,
and then click OK.
4. Select a trust level as follows:
v Trust all SiteProtector trusts the server and does not try to validate the
certificate
v First time trust SiteProtector trusts the first certificate it receives from the
server and stores this certificate locally. The client uses this certificate to
validate all future communication with this server.
v Explicit trust The certificate for the server must reside in a local
SiteProtector directory before the agent can initiate communication with
the server. Typically, the certificate is transferred to the client outside the
standard communication channels.
5. Click OK.
Update Settings
Tip: Consider inheriting this policy from the parent to ensure all agents are
running the same version and build of the software and that all agents are
installed in the same directory.
1. If you do not see this policy, expand Policy Types Not Created, right-click
the policy, and then select New Policy.
2. In the Agent Version list, select the agent version and build.
3. In the Install path box, type the location where the agent will be installed
on the host system.
Security Events
see Chapter 4, “Protecting against intrusions,” on page 33
System Integrity Monitoring
see “Auditing operating system logs” on page 59
Text Log Monitoring
see “Auditing text logs” on page 70
9. Click the Save icon.
10. From the Save Policy Version window, enter a Comment and then select the Deploy This New
Version check box.
11. Click OK. The Deploy Policy window opens.
12. Click Targets.
13. Select the applicable group to which you want to deploy the policy.
14. Click OK.
Chapter 2. IBM Security Host Protection for Windows Server agents deployment
21
15. Do one of the following:
v If you are deploying policies using an agent build, go to “Creating and Deploying an Agent
Build”
Note: If you are deploying an agent for the first time, you must use the agent build mechanism to
deploy the agent.
v If you are deploying policies using the heartbeat mechanism, do one of the following:
– Wait for the agents to send a heartbeat to SiteProtector
– Select the applicable group, and then select Action > Refresh Agent to force the agents to send
a heartbeat to SiteProtector immediately
Creating and Deploying an Agent Build
You can install IBM Security Host Protection for Windows agents using an agent build created from
SiteProtector. This section provides the process and procedures for creating an agent build and for
installing IBM Security Host Protection for Windows, Version 2.x using that agent build.
What is an agent build?
An agent build is a preconfigured installation package that includes the following software and
user-defined information:
v The actual IBM Security Host Protection for Windows software that the build installs on the computer
v Information about how the agent should communicate with SiteProtector, including the IP address of
the Agent Manager it reports to and the account name and password it should use when it contacts the
Agent Manager
v Policies that define the protection provided by the agent
Prerequisites
Review the appropriate deployment scenario checklists before using the information in this section to
create and deploy an agent build.
Reference: “Deployment scenario checklists” on page 6
Topics
“Generating an Agent Build” on page 23
“Installing an Agent” on page 24
22
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Generating an Agent Build
About this task
An agent build is a preconfigured installation package that you can use to install IBM Security Host
Protection for Windows agents. The agent build contains the agent software and the policies you
configured to define the behavior of the agent.
Generating an Agent Build
Procedure
1. Select Object > New Tab > Agent.
2. In the navigation pane, right-click the group, and then select Generate Agent Build.
The Generate Agent Build window opens.
3. Click the Details icon.
4. In the Command Details section, select Host Protection for Windows Server from the Agent Type
list.
5. Select the Agent Version.
6. Click the Build icon.
7. In the Agent Manager list, do one of the following:
v Verify that the Agent Manager listed is the correct Agent Manager for this group
v Select an Agent Manager from the Agent Manager list
8. Type a Description for this build.
9. Click OK.
Checking the agent build generated successfully
Procedure
1. In the navigation pane, right-click the group, and then select Properties.
The group properties page opens.
2. In the left pane, click Command Jobs.
The Command Jobs pane opens.
3. Sort the Command Jobs by the Start Time, and then identify the latest agent build in the Command
column.
4. Click the agent build command job.
The details of the command job are displayed in the lower pane.
5. Confirm that the Status is Completed.
Note: If the status is either Pending or Processing, then press F5 to refresh the status.
Important: Do not distribute the agent build until the status is Completed.
Chapter 2. IBM Security Host Protection for Windows Server agents deployment
23
Installing an Agent
About this task
After you create the agent build, you can access the build from the host system and install the agent on
the server.
Procedure
1. On the computer where you want to install the IBM Security Host Protection for Windows agent, start
Internet Explorer.
2. Type the URL to the SiteProtector system by using the following format:
http://Agent_Manager_IP_address:8085
The Agent Manager Available Downloads web page opens.
3. Select the agent build you want to install, and then click Download.
4. Save the agent build program file to the local system, and then double-click the file to begin the
installation process.
What to do next
After you install IBM Security Host Protection for Windows, you can use the following list to verify that
the installation completed successfully:
Look here...
To verify that...
System tray
The IBM Security Host Protection for Windows icon is glowing (if you installed
the local user interface)
Task manager
v On Windows Server 2003 32-bit systems, blackd, BlackICE, RapApp, and
phService are running
v On Windows Server 2003 64-bit systems, blackd, BlackICE, and phService are
running
v On Windows Server 2008 and Windows Server 2008 R2, blackd, BlackICE,
and phService are running
IBM Security Host Protection for
Windows
The agent name, the IP address, and an Active status are showing in the Agent
view
Services
v On Windows Server 2003 32-bit systems BlackICE, RapApp, and IBM Security
Protection services are started
v On Windows Server 2003 64-bit systems BlackICE, and IBM Security
Protection services are started
v On Windows Server 2008 and Windows Server 2008 R2, BlackICE, and IBM
Security Protection services are started
Events tab in the local UI
An alert indicates agent-startup was successful (if you installed the local user
interface)
If no services and processes start, look in the Windows directory for the AgentUpdateYour_System_Name.log for possible issues with the installation.
24
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 3. Firewall configuration
The firewall is the first line of defense against a network-based attack on your computer. You can create
firewall rules as part of the central policy, as part of the local policy (if your policy allows for local
management), or automatically when the agent detects a threat. IPv6 is supported on Windows Server
2008 and later.
Topics
“Key concepts for firewall”
“Protection levels” on page 26
“Firewall blocking” on page 27
“Firewall rules” on page 28
“Opening a port through the firewall” on page 31
“Advanced firewall configuration” on page 32
Key concepts for firewall
The firewall is the computer's first line of defense against a network-based attack.
Disabling the agent firewall
Note: This feature applies to Server Protection for Windows Version 2.1 and later.
If the Microsoft Windows firewall is running, the IBM Security Host Protection for Windows agent
disables the Windows firewall to avoid conflicts, and uses its own firewall instead. If the agent is stopped
or fails, the agent restores the Microsoft Windows firewall until the agent is restarted. Any configuration
settings for the Microsoft Windows firewall are preserved.
IPv6
IBM Security Host Protection for Windows can detect IPv6 attacks. Any special considerations related to
IPv6 support are noted in the online help.
Note: In the Group Settings policy on the Agent Manager List tab, the Host Name does not support IPv6
addresses.
Level of configuration
The level of configuration required for the firewall varies depending on the level of protection you want
to enforce. Choosing a protection level quickly implements perimeter security. Defining firewall rules
provides more granular control.
© Copyright IBM Corp. 2005, 2012
25
Protection levels
A protection level is a set of predefined firewall rules that provide a certain level of security. Protection
levels allow or restrict access to the system based on port and protocol. For example, at the Paranoid
level, the agent blocks incoming traffic on all TCP and UDP ports.
Important: Protection levels do not block outbound traffic. To block outbound traffic, you must define
outbound firewall rules.
Protection level settings
The following table describes the predefined protection levels:
Level
Description
Paranoid
Blocks all TCP ports and all UDP ports; blocking all
unsolicited inbound traffic.
Nervous
Blocks all TCP ports and blocks UDP ports 0-1023. This
effectively blocks all unsolicited inbound traffic except
for some interactive content on Web sites (such as
streaming media and other application-specific Internet
usage).
Cautious
Blocks TCP and UDP ports 0-1023, effectively blocking
unsolicited network traffic that accesses the operating
system and networking services.
Trusting
All ports remain open and unblocked allowing all
inbound traffic.
Considerations
Keep the following in mind when you set up protection levels:
v Ports 137 and 138 accept network traffic when Network Neighborhood lookup is enabled and they
block traffic when the feature is disabled.
v Ports 139 and 445 accept network traffic when file sharing is enabled and they block traffic when file
sharing is disabled.
v For computers running Windows Server 2000, Windows Server 2003, or Windows Server 2008 both
ports 139 and 445 are used for the SMB file sharing protocol. Therefore, when default firewall settings
from the management server are applied to the local agent, accept rules for both port numbers are
automatically created.
v If configuration sharing is enabled, the end user can enable or disable Network Neighborhood lookup
and file sharing through the local agent console. The Protection Level may also be set through the local
console.
Navigation
Locate the policy you want to edit and then click Firewall.
26
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Firewall blocking
A firewall can reduce, but not eliminate, exposure introduced by networked hosts. Firewall technology
can help prevent attacks by limiting access to ports, IP addresses or ranges of IP addresses required for
legitimate system activity.
A firewall is one of several approaches you can use to block malicious traffic. See Chapter 4, “Protecting
against intrusions,” on page 33 for a different approach.
Automatic blocking
When the agent detects an attack that poses an immediate threat to the system, it can automatically
instruct the firewall to block the intruder. By default, automatically created blocks expire after 24 hours.
Because the firewall controls transmission at the TCP/IP level, intruders cannot get around a block from
the agent. Automatically created blocks have precedence over manually created firewall rules.
Manual blocking
Security managers can manually configure the firewall to block or accept traffic from any IP address or
TCP/UDP port for a period of time or permanently.
Blocking
The agent can block traffic identified by IP address, port, or protocol:
v For rules based on an IP address, the agent checks the source IP of inbound communications, and the
destination IP for outbound communications.
v For rules based on a port number or protocol, the agent checks the port the communication is destined
for.
Protocol numbers refer to the 8-bit field in the IP header that specifies protocols such as TCP, UDP, or
ICMP.
Using firewall blocking for information access
You can use firewall rules to create access policies that prevent users from accessing applications
advertised on specific ports. These access policies can block inbound or outbound traffic to these ports
based on IP addresses or ranges of IP addresses.
Chapter 3. Firewall configuration
27
Firewall rules
You can create firewall rules to block inbound, outbound, or bidirectional traffic based on IP type or
address, port number, or ICMP type or code. In addition, the agent can accept or reject inspected packets
based on the source IP address.
Types of firewall rules
The following table lists the basic types of firewall rules:
Filter Option
Description
IP type
An IP Type rule gives you an easy way to block a certain type of IP traffic.
For example, you can block all UDP traffic by selecting UDP [17] from the IP Type list,
Reject from the Action list, and BOTH from the Direction list.
IP address
An IP address rule specifies whether an agent should accept or reject traffic to or from a
particular IP address or range of addresses.
UDP port
A UDP port rule specifies whether an agent should allow or block traffic through a
particular User Datagram Protocol (UDP) port. You can associate the port with an IP
address or address range.
TCP port
A TCP port rule specifies whether an agent should allow or block communication through
a particular Transmission Control Protocol (TCP) port. You can associate the port with an IP
address or address range.
ICMP type or code
An Internet Control Message Protocol (ICMP) rule specifies whether an agent should allow
or block traffic of a particular packet type or code. You can associate the packet type or
code with an IP address or address range.
IPv4 address expression examples
The <n> can be either hex or decimal number in a range from 0 to 255. All hex numbers must have a 0x
prefix.
Single address: n.n.n.n
Address range, where last value is greater than first: n.n.n.n - n.n.n.n
IPv6 address expression examples
The value for <n> must be a hexadecimal digit (0-F). Any four-digit group of zeroes within an IPv6
address may be reduced to a single zero or omitted.
Single address: nnnn:nnnn:nnnn:nnnn:nnnn:nnnn:nnnn:nnnn
Address range, where last value is greater than first: nnnn:nnnn:nnnn:nnnn:nnnn:nnnn:nnnn:nnnn nnnn:nnnn:nnnn:nnnn:nnnn:nnnn:nnnn:nnnn
Navigation
Locate the policy you want to edit and then click Firewall > Firewall Rules.
28
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Firewall precedence
Use the Firewall Rules tab in the IBM Security Host Protection for Windows agent's Firewall policy to
add firewall rules based on IP type, IP address, UDP port, TCP port, and ICMP type and code.
Note: Any Intrusion Prevention rules takes precedence over firewall rules.
Order of precedence
The agent applies firewall rules to network traffic based on traffic type and action requested.
The agent processes rules in the order indicated in the following table:
Priority
Traffic Type
Action
1
IP Address
Accept
Protocol
IP Address and Port/ICMP/Protocol
combination
2
IP Address
Reject
Protocol
IP and Port/ICMP/Protocol combination
3
Port/ICMP
Reject
4
Port/ICMP
Accept
5
All others
Accept
Examples of firewall precedence
This topic provides example scenarios to illustrate how you can make the application order of firewall
rule precedence in the IBM Security Host Protection for Windows agent work for you.
Scenario 1
You want to block all outbound UDP packets except for outbound UDP DNS packets, and you have the
following UDP Rules in place:
v UDP Rule 1: All, 53, ACCEPT, OUTBOUND
v UDP Rule 2: All, All, REJECT, OUTBOUND
Important: If the IP address setting for a UDP Rule or a TCP Rule is configured as “All” (and not
specified numerically), the firewall interprets the rules as a “Port/ICMP” Traffic Type.
In this scenario, Rule 1 would be priority 4 and Rule 2 would be priority 3. As a result, all outbound
UDP packets would be rejected because a priority 3 rule (Rule 2) is processed before a priority 4 rule
(Rule 1).
Suggested approach
Use the following rules:
v UDP Rule 1: 0.0.0.0-255.255.255.255, 53, ACCEPT, OUTBOUND
v UDP Rule 2: All, All, REJECT, OUTBOUND
Chapter 3. Firewall configuration
29
While this approach does not change the logic of the rule (0.0.0.0-255.255.255.255 is the same as “All”), it
does change the priority of UDP Rule 1 from priority 4 to priority 1 because the firewall interprets the
numeric IP addresses as an “IP Address” Traffic Type not a “Port/ICMP” Traffic Type. This syntax change
allows outbound DNS queries while blocking all other outbound UDP packets.
Scenario 2
You want to block unused service ports on a computer. Your first thought might be to do both of the
following actions:
v Use “IP accept” rules (Priority 1) to allow access to the computer from your corporate network
v Use “IP and port reject” rules (Priority 2) to block unused ports
This approach is not effective because of the order in which the agent applies firewall rules. The “IP
allow” rules are processed first, thereby allowing all IP traffic through. All packets would match the “IP
allow” rules; therefore, the “IP and port reject” rules are never used.
Suggested approach
Use both of the following rules:
v “IP Port accept” rules (Priority 1) for all services that you want to allow
v “IP reject” rules (Priority 2) on all others
This approach is effective because the “IP Port accept” rules cover only services that you specifically
allow. Not all packets would match these rules, therefore the “IP reject” rules are applied, denying access
to all others.
Managing conflicts between firewall rules
Use the Firewall Rules tab in the IBM Security Host Protection for Windows agent to add firewall rules
based on IP type, IP address, UDP port, TCP port, and ICMP type and code.
A conflict exists when two or more rules of the same type specify different treatments for any parameter.
The agent resolves such conflicts by applying a special set of priority levels that are internal to a rule
type.
30
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Table 1. Criteria for arriving at an internal rule set without overlaps
Priority
Element
Explanation
1
Precedence
Depending on the level of
configuration sharing in effect, rules
received from a management console
may take priority over locally-created
rules or vice versa.
2
Time
If two or more conflicting rules have
the same precedence, the one with
the later timestamp takes priority.
3
Position
If conflicting rules have the same
precedence and timestamp, the latest
(lowest) entry in the file takes
priority.
Opening a port through the firewall
Some applications may require ports to be manually opened so that the application can work correctly.
As the agent analyzes outbound packets, it can dynamically open the necessary ports for certain
applications. This feature allows you to keep the agent set to a more secure protection level without
manually opening and closing ports in the firewall.
Automatic port opening for certain applications and protocols
Agents can automatically open ports for the following applications and protocols:
v Real Audio
v
v
v
v
v
v
Microsoft Windows Media Player
RealPlayer
Internet Relay Chat (IRC)
Microsoft NetMeeting
DNS
FTP
v UDP
Disabling automatic port opening
Each application has its own custom parameter that controls whether port opening is turned on or off. By
default, port opening is turned on. You can turn it off by configuring the parameter for the specific
application in the Advanced Configuration tab.
Example: Add the tunnel.ftpserver parameter with a value of disabled to prevent port opening for FTP
servers.
Reference: See the Custom Parameters Help for the list of parameters and values.
Opening ports for UDP traffic
The agent monitors all outgoing UDP frames and allows incoming frames from the same port and IP
address for 10 seconds. Opening ports for UDP traffic enables the following protocols to operate properly
in all protection modes:
Chapter 3. Firewall configuration
31
Protocol
Ports
DNS
53, 137
LDAP
389
Kerberos
88 and 464
SMTP
123
pcANYWHERE client
22, 5632
IPSEC
500
Manually opening a port
For some applications that use application ports, you must manually open ports when the agent is set to
the Nervous or Paranoid protection level.
Advanced firewall configuration
The IBM Security Host Protection for Windows agent supports a set of custom parameters that you can
use to fine-tune how the agent firewall handles intrusions.
Each parameter consists of the following elements:
v Parameter name
v Expected value
v Description
Important: Do not edit the parameters file manually. The Custom Firewall Parameters section of the
policy provides an easy to use interface for adding custom parameters.
Navigation
Locate the policy you want to edit and then click Firewall > Advanced Configuration.
Reference: See the Custom Parameters Help for detailed information about firewall parameters.
32
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 4. Protecting against intrusions
The IBM Security Host Protection for Windows agent provides preemptive protection by examining a
wide variety of protocols, including TCP, UDP, ICMP, and IGMP. The agent parses the protocols, analyzes
them heuristically and deterministically, and analyzes them for harmful content.
Protection provided
The agent prevents the following from being used to attack a system:
v Known exploits
v Unknown exploits against known vulnerabilities
v Unknown exploits that attempt to exploit a system by abusing a specific protocol
Topics
Understanding security events
Intrusion prevention or intrusion detection
Security events
“Trusted addresses” on page 36
“Tuning the agent” on page 37
“Event filtering” on page 39
“Advanced security event configuration” on page 40
“Back tracing” on page 40
“Packet logging” on page 41
“Evidence Logging” on page 42
© Copyright IBM Corp. 2005, 2012
33
About security events
This topic describes how the Security Events policy in the IBM Security Host Protection for Windows
agent works.
Where most other protection products look at exploits through static signatures, Security Events
understands the underlying protocol and the associated vulnerabilities.
Security Events analyses the data stream. If the data contains a packet that is not typical for a particular
protocol, Security Events flags the packet as suspicious, and reports or blocks it. If there are specific
vulnerabilities associated with a specific type of traffic, Security Events identifies the traffic as malicious
and protects the computer.
How security events work
The IBM Security Host Protection for Windows agent analyzes packets as they enter or exit the system.
Based on this analysis, the agent responds in one of the following ways:
v If the agent detects no malicious activity, it lets the packets continue unhindered.
v If the packets contain an attack that is not immediately threatening, the agent triggers a report to the
SiteProtector System and the local user console (if installed). The agent can also log the traffic to
evidence or packet logs if configured to do so.
v If the attack poses an immediate threat to the computer, the agent sends a report to the SiteProtector
System and to the local console (if installed). In addition, the agent stops subsequent incoming packets.
The firewall can block the invading packet using any combination of the following methods:
v Blocking the source IP address
v Blocking the destination port number
v Issuing a TCP reset
Event filter and bypass filter interaction
The agent processes bypass filters before it processes event filters, so it is possible to enter a bypass filter
that makes an event filter redundant. Although the effect on the system is similar (traffic can circumvent
the protection offered by security event rules), you might not see the expected behavior if you later clear
the event filter. That is, when you clear the event filter, you would expect the security event rules to
provide protection against malicious traffic, but, because the bypass filter is still in effect, packets might
still be circumventing the protection offered by security event rules.
The Protocol Analysis Module (PAM)
The Protocol Analysis Module supplies the information that agents use to protect the host computer
against intrusions. PAM uses a database that stores the signatures and handles specifications for a
comprehensive list of intrusions.
34
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Intrusion prevention or intrusion detection
You can control whether the agent operates in intrusion prevention (IPS) mode or intrusion detection (IDS)
mode.
Choosing IPS mode or IDS mode
The following table describes the settings for IPS and IDS modes:
To use...
Select...
Results
Intrusion prevention mode
Block and alert
When the agent detects an intrusion,
it blocks the intrusion and alerts you.
Intrusion detection mode
Alert only
When the agent detects an intrusion,
it alerts you, but does not block the
intrusion.
Intrusion detection is useful if you
want to see what type of intrusions
are occurring. Some security
managers use this approach to
monitor intrusions before they set up
blocking.
Important: If you select Alert only,
no events detected by rules
configured in the Security Events tab
are blocked, even if “block” is the
default action for the rule.
Navigation
Locate the policy you want to edit and then click Security Events.
Security events
The Security Events tab lists hundreds of rules that can protect your system against attacks.
You can view security rules and edit their settings individually.
You can control the following settings for each security check:
v Whether the check is enabled or disabled
v Severity level
v Whether the event is blocked or only reported
Important: The “Block” settings for events are enforced only when the agent is running in intrusion
prevention mode. In intrusion detection mode, intrusions are only reported, regardless of the block status
shown for events. See “Intrusion prevention or intrusion detection” for information.
Navigation
Locate the policy you want to edit and then click Security Events > Security Events.
Chapter 4. Protecting against intrusions
35
Descriptions for security event columns
This topic describes the column headings for the Security Events tab in the IBM Security Host Protection
for Windows agent's Security Events policy.
Table 2. Column descriptions for Security Events
Column Name
Description
Enabled
Indicates whether the security check is enabled
Tag Name
Name of the security check
Severity
Current severity level of the event. Severity levels are
High, Medium, and Low
Protocol
Communication protocol used by the attack or security
event
Block
Indicates whether the security event is blocked
XPU
X-Press Update version for this event
Block Overridden
Indicates whether the agent uses the default blocking
behavior for this event, or whether the default blocking
behavior has been overridden
Severity Overridden
Indicates whether the severity for this event is set to the
default, or whether the default has been overridden
Issue ID
The unique identifier that X-Force has assigned to this
security event
Blocking Type
Type of block the agent uses to protect against this
security event
Check Date
Date the check for this event was implemented
Trusted addresses
When you specify a trusted IP address, the IBM Security Host Protection for Windows agent ensures that
traffic associated with that addresses is not blocked by the security event rules. Trusting ensures that
incoming traffic from useful or helpful addresses is not blocked automatically.
Important: Consider trusting or ignoring only those addresses that do not pose a threat to your system.
Keep in mind that an intruder can “spoof”, or fake, the IP addresses of an internal system.
Note: Event filters give you more granular control over trusted addresses. The trusted addresses feature
is provided to support trusted addresses configured in earlier agent versions
Interaction between firewall rules and trusted addresses
Firewall rules are processed before trusted address filters. If you enter a reject firewall rule that conflicts
with a trust filter, the traffic will be rejected by the firewall rule.
Interaction among filters
The agent supports bypass filters, event filters, and trusted address filters.
v Bypass filters allow packets from certain IP addresses to bypass analysis by the firewall and the
security event rules.
v Event filters ensure that traffic associated with useful or helpful addresses is not blocked by security
event rules.
v Trusted address filters ensure that traffic associated with useful or helpful addresses is not blocked by
security event rules.
36
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Because there are several ways for the agent to filters traffic, it is possible to enter filters that make other
filters redundant. Although the effect on the system is similar (traffic is allowed to circumvent the
protection offered by the agent), you might not see the expected behavior if you later clear a filter. If you
are unaware that there are redundant filters configured, you might expect the agent to resume processing
certain traffic after you clear a filter; however, there might still be a filter configured that allows certain
traffic to circumvent the protection offered by the agent.
References: See “Event filtering” on page 39 and Chapter 9, “Configuring bypass filters,” on page 73. See
“Traffic seems to be bypassing analysis” on page 86 for troubleshooting advice.
IP address or IP range
You can create trust rules for individual IP addresses or for a range of IP addresses.
Navigation
Locate the policy you want to edit and then click Security Events > Trusted Addresses Detail.
Tuning the agent
By default, the agent is configured to report and block many types of intruder activity, including port
scans, worm infection attempts, and general suspicious activity. Depending on your network, this default
can produce any number of events, from just a few to many.
Reducing the number of events you see
As a security manager, you might or might not be interested in monitoring port scan blocks and other
less critical events. You can configure trust rules to help reduce the number of events you see.
The following table describes trust rules:
Rule Type
Description
trust.pair
Allows and ignores only specific events from specific
computers.
The agent does not allow any other events from trusted
computers. Similarly, the agent does not allow the
trusted events when they originate from a computer that
is not trusted.
Important: When implemented correctly, this type of
rule is the safest trust rule you can use.
trust. issue
Allows and ignores events you specify, regardless of
where they originate.
trust.myself.issue
Allows and ignores only specific events that originate
from the computer.
trust.myself
Allows and ignores all events that originate from the
computer.
CAUTION:
Use this type of rule carefully! If the computer becomes
infected, the agent does not report the outgoing attacks
to SiteProtector.
Chapter 4. Protecting against intrusions
37
Rule Type
Description
trust.address
Allows and ignores all attacks from one or more IP
addresses.
CAUTION:
Use this type of rule carefully! If you create a
trust.address rule, the agent can no longer protect the
computer if the trusted computer becomes infected by a
virus or compromised by an attacker. The agent does
not report the attack to SiteProtector.
Where to add rules
Add these rules using the Advanced Configuration tab in Security Events policy. See “Advanced security
event configuration” on page 40.
Using SiteProtector to reduce the number of events you see
To further reduce the number of events you see, use the SiteProtector console to do the following actions:
v Create event filtering rules in SiteProtector that allow agents to block less severe attacks locally, but
prevent these events from being saved to the SiteProtector database by filtering events by severity at
the Agent Manager.
v Save all IPS events to the SiteProtector database, but create exceptions to automatically hide certain
types of events from the analysis view and reporting.
Raising the visibility of an event
If you want to escalate the visibility of an event, you have the following options:
v Create an incident in SiteProtector.
v Request alerts (such as email messages or pages) through central responses in SiteProtector.
False positives
If you find that the agent is falsely identifying some types of network traffic in your environment, report
the issue to IBM Security Solutions. For information about submitting reports, see “Reporting a false
positive” on page 82.
38
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Event filtering
An event filter ensures that traffic associated with useful or helpful addresses is not blocked by rules
configured on the Security Events tab. An event filter can reduce the number of alerts displayed in the
Console and the number of false positives detected by the agent.
Important: Consider filtering events for only those addresses or events that do not pose a threat to your
system. Keep in mind that an intruder can “spoof”, or fake, the IP address of an internal system.
Example
A known user pings you frequently and the agent sends an alert when it detects the ping flood event on
the host. As you know this is a harmless event, you configure an event filter for this IP address and this
event. When the agent detects the ping flood event from any IP addresses defined in the event filter, it
does not block the traffic or send an alert to the console.
How it works
After traffic passes through the firewall, the agent processes the rules configured on the Security Events
tab against the traffic. If the traffic triggers a security event rule, the agent would typically take protective
action and send an alert to the console to notify you of the potential threat to your system. If, however,
you have configured an event filter for this type of event from this IP address, the agent takes no
protective action, sends no alert to the console, and allows the packet to continue (even if the packet
contains malicious content).
Interaction among filters
The agent supports bypass filters, event filters, and trusted address filters.
v Bypass filters allow packets from certain IP addresses to bypass analysis by the firewall and the
security event rules.
v Event filters ensure that traffic associated with useful or helpful addresses is not blocked by security
event rules.
v Trusted address filters ensure that traffic associated with useful or helpful addresses is not blocked by
security event rules.
Because there are several ways for the agent to filters traffic, it is possible to enter filters that make other
filters redundant. Though the effect on the system is similar (traffic is allowed to circumvent the
protection offered by the agent), you might not see the expected behavior if you later clear a filter. If you
are unaware that there are redundant filters configured, you might expect the agent to resume processing
certain traffic after you clear a filter; however, there might still be a filter configured that allows certain
traffic to circumvent the protection offered by the agent.
References: See “Trusted addresses” on page 36 and Chapter 9, “Configuring bypass filters,” on page 73.
See “Traffic seems to be bypassing analysis” on page 86 for troubleshooting advice.
Navigation
Locate the policy you want to edit and then click Security Events > Event Filters.
Chapter 4. Protecting against intrusions
39
Advanced security event configuration
The IBM Security Host Protection for Windows agent supports a set of custom parameters that you can
use to fine-tune how agents detect and prevent intrusions.
Each parameter consists of the following elements:
v Parameter name
v Expected value
v Description
Important: Do not edit the parameters file manually. The Advanced Configuration tab in the policy
provides an easy to use interface for adding custom parameters.
Navigation
Locate the policy you want to edit and then click Security Events > Advanced Configuration.
Reference: See the Custom Parameters Help for detailed information about IPS parameters.
Back tracing
Use the back trace feature to trace intrusions from other computers. The agent can trace the source of
network traffic by analyzing information in the header of a packet that triggers an event. Back tracing
also tries to identify other names for the intruder's computer and the hardware address, which makes
viewing event information easier for you.
Where are back trace results?
The results of the back trace are available in the following locations:
v Event Details in SiteProtector
v The Intruder tab in the local console (if installed on the computer)
Type of tracing
The agent can perform the following types of tracing:
v Indirect tracing
v Direct tracing
You can configure an agent to use both direct and indirect tracing.
Indirect tracing
Indirect tracing does not make contact with the intruder's system, but identifies an intruder by querying
such sources as NetBIOS and the DNS database. This option returns the intruder's host name and lists it
in the Events log.
Direct tracing
Direct tracing follows the network path back to the intruder's system to find out the computer name. In
general, direct traces gather more reliable information than indirect traces, but direct traces can alert the
intruder of the agent's activity.
Navigation
Locate the policy you want to edit and then click Security Events > Back Tracing.
40
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Packet logging
The agent can store information about all system traffic in log files. When you enable packet logging, the
agent generates and maintains a set of packet log files.
Important: Before you use this option, decide how you want to collect the logs from computers. To
collect the logs, you must have local access rights to the computer, or you must have someone at the
computer send the logs to you.
How is packet logging different from evidence logging?
In contrast to evidence logging, which only collects information about intrusion attempts, packet logging
captures information about all system traffic. Therefore, packet logs can grow very large and consume a
great deal of hard disk space.
Packet log files are filled until a maximum size is reached, and then the agent generates a new file. When
the maximum number of files is reached, the agent overwrites the first log file with a new file. You can
configure maximum file size and maximum number of files as part of the policy.
Locating packet logs
The agent stores packet logs in the directory where it is installed. The file prefix is log by default, or the
prefix value you set. The file extension for all packet log files is .enc.
Tip: When sorted alphabetically, all packet log files are grouped together according to the prefix you
define.
Viewing packet log information
If you want to view the packet logs, you can use a trace file decoding application to open the packet log
file. You can set a password to protect the log files. No one can access the log files without entering the
password.
Note: You can purchase trace file decoders on the Internet.
Navigation
Locate the policy you want to edit and then click Security Events > Packet Log.
Chapter 4. Protecting against intrusions
41
Evidence Logging
When the agent identifies an incoming packet as a security event, the packet is captured and encoded
into an evidence file. Evidence files can provide proof of an intrusion and provide some indication of the
intruder's intentions.
Important: Before you use this option, decide how you want to collect the logs from computer. To collect
the logs, you must have local access rights to the computer, or you must have someone at the computer
send the logs to you.
How is evidence logging different from packet logging?
In contrast to packet logging, which collects information about all network traffic, evidence logging
captures only information about intrusions. The agent can capture a single packet of an incoming
communication that it identifies as a security event. If you need to, you can use the evidence in this file
for an investigation.
Locating evidence logs
Evidence logs are stored in the directory where the agent is installed. The file prefix is evd by default, or
the value you set on the Evidence Log tab. The file extension for all log files is .enc.
Tip: When sorted alphabetically, all evidence log files are grouped together according to the prefix you
define.
Viewing evidence log information
If you want to view the evidence logs, you can use a trace file decoding application to open the packet
log file.
Note: You can purchase trace file decoders on the Internet.
For help with analysis
You can send your log files to Technical Support for analysis. For help with interpreting logs, contact IBM
Security Solutions Technical Support.
Navigation
Locate the policy you want to edit and then click Security Events > Evidence Log.
42
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 5. Protecting against buffer overflow exploits
The IBM Security Host Protection for Windows agent can prevent intruders from using a buffer overflow
to exploit a server. The BOEP component inspects memory as it is extracted in the execution space,
helping to detect and prevent both known and unknown attacks that target unknown vulnerabilities on
the host system.
Note: Buffer Overflow Exploit Prevention is not currently supported on any 64-bit system, Windows
Server 2008, or Windows Server 2008 R2.
Topics
“Buffer overflow exploit prevention (BOEP)”
“Understanding buffer overflow exploit prevention (BOEP)” on page 44
Buffer overflow exploit prevention (BOEP)
Use the BOEP page in the IBM Security Host Protection for Windows agent to monitor for programmatic
behaviors that indicate that a buffer overflow is occurring. The agent needs no prior knowledge of
specific exploits or vulnerabilities to protect computers from buffer overflow exploits.
Note: Buffer Overflow Exploit Prevention is not currently supported on any 64-bit system, Windows
Server 2008, Windows Server 2008 R2, Windows Vista, or Windows 7.
What to expect from a buffer overflow
After a buffer overflow attack, the server operator may need to restart the target application to recover. If
the compromise affects a core service like MS-RPC or LSASS, the server operator may need to restart the
server, and the agent may not be able to log the event.
Important: Until you address the root cause or vulnerability, these side effects of an attack may become
common occurrences.
What BOEP does not prevent
The BOEP component prevents exploits that attempt to use buffer overflows to run; it does not prevent
buffer overflows themselves. Vulnerability to buffer overflows is a characteristic of some applications. If a
buffer overflow occurs, the computer may need to be restarted.
Default monitoring
When BOEP is enabled, the agent automatically protects the most commonly exploited applications from
buffer overflow exploits. You have the option to include additional folders to monitor additional
applications. Keep in mind that adding folders may increase the risk of false positives. Specify inclusions
by referencing folders or files; specify exclusions by referencing applications.
Handling false positives
When you determine that a reported buffer overflow event is a false positive, you may want to add the
associated application to your excluded applications list. Keep in mind, however, that you may risk
future buffer overflow attacks in this associated application.
© Copyright IBM Corp. 2005, 2012
43
BOEP and Data Execution Prevention (DEP)
By default, Data Execution Prevention (DEP) is enabled on Windows 2003 Server. DEP may block certain
buffer overflow exploits before the BOEP component can analyze them and sent an alert. If you want the
IBM Security Host Protection for Windows agent to monitor for buffer overflow exploit events, disable
DEP.
Understanding buffer overflow exploit prevention (BOEP)
Buffer Overflow Exploit Prevention (BOEP) can stop intruders from using a buffer overflow to exploit an
end user's computer. BOEP monitors for programmatic behaviors that indicate that an overflow is
occurring. When BOEP is enabled, the agent needs no prior knowledge of specific exploits or
vulnerabilities to protect computers from buffer overflow exploits.
The importance of layered protection
Buffer overflow exploit prevention should be the last line of defense against packet-based attacks in a
layered approach that includes a personal firewall and intrusion prevention.
A personal firewall blocks known and unknown attacks against the ports and services you do not need.
Intrusion prevention filters out known and unknown attacks against known vulnerabilities.
Buffer overflow exploit prevention technology is not designed to be the first line of defense from an
attack, as it allows some degree of collateral damage. In an attack, the buffer overflow succeeds;
therefore, the targeted process memory may be corrupted and result in a denial of service.
What it does
The BOEP component prevents exploits that attempt to use buffer overflows to run; it does not prevent
buffer overflows themselves. Vulnerability to buffer overflows is a characteristic of some applications. The
IBM Security Host Protection for Windows agent monitors system calls commonly used by malicious
code. If it detects malicious code attempting to make such a system call, the agent blocks the call and
prevents the intended operation.
Depending on the response you configure the agent to take if it detects a buffer overflow exploit, the
computer may need to be restarted. For example, if the agent is configured to terminate the process
where the buffer overflow was detected, and the process is critical to system operation, the system may
shutdown.
How it works
BOEP protects host computers from buffer overflow attacks through known and unknown vulnerabilities.
As a general rule, a computer should never execute code from writable areas of system memory. By
watching the use of Stack and Heap system memory, the agent identifies when a buffer overflow has
succeeded and prevents the attack's payload from running. BOEP stops the worm from propagating and
prevents the attacker from remotely executing code on the local computer.
Definitions: buffer overflow and buffer overflow exploit
A buffer is a section of memory used for storing data until the process that needs the data is ready to
receive it. Some intruders attempt to send more data to the buffer than it can handle, causing a buffer
overflow.
Intruders may then be able to execute arbitrary commands in a program with administrator permissions,
effectively taking control of the remote computer. This activity is the buffer overflow exploit.
44
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Protection for common applications
When buffer overflow exploit prevention is enabled, the agent automatically protects against buffer
overflow exploits in common applications.
For the most complete, up-to-date list of monitored applications, visit the Technote on the IBM Support
portal at https://www.ibm.com/support/docview.wss?uid=swg21468637. Search for KBA #2736.
IBM Security may update this list in security updates, and you can add additional folders or files for
monitoring.
Protection modes
You can run buffer overflow exploit prevention in one of the following protection modes:
Setting
Description
Fail system call and send alert
Blocks the suspicious system call, but allows the program
to continue running. Sends an alert to the console.
Terminate process and send alert
Ends the program and sends an alert to the console.
Alert only
Allows the suspicious call and the program to continue
running, but sends an alert to the console.
Tip: When you first start using BOEP, use the “Alert only” setting. This setting, sometimes called
“simulation mode”, can help you identify false positives without interrupting normal business operations.
Chapter 5. Protecting against buffer overflow exploits
45
46
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 6. Enforcing antivirus compliance
With antivirus compliance enabled, the agent can enforce the requirement for antivirus software on a
computer.
You can control which antivirus software version must be running on the host system and define the
actions the agent should take if the computer is not running the required software.
Level of configuration
Antivirus compliance requires only minimal configuration, including selecting the required antivirus
program, setting compliance parameters, and providing message text.
Topics
“Understanding antivirus compliance”
“Configuring antivirus compliance” on page 48
“Advanced antivirus compliance configuration” on page 49
Understanding antivirus compliance
The IBM Security Host Protection for Windows agent can enforce antivirus software compliance on
computers. You can control which antivirus software is required and set actions to be taken if a computer
is not in compliance.
How antivirus compliance works
Each time the system starts and periodically thereafter, the agent runs a series of checks to ensure that the
computer meets the antivirus compliance standard specified in the policy. If the computer does not meet
this standard, the agent takes the following actions:
v Sends an alert to the console
v Limits network access (if you have selected that option); the agent can block outbound connections to
IP addresses listed in the IP Ranges tab in Group Settings
Supported antivirus software
The agent can enforce the presence of the following antivirus software:
v Symantec/Norton Antivirus
v McAfee Antivirus
If the required antivirus software is not running on the computer, the computer is out of compliance. The
agent responds by taking the action you have specified.
Important: If you enable antivirus compliance for a group, each computer in that group must use the
same antivirus program. If a computer is running an antivirus application other than the one specified in
the policy, the agent may block the computer's access to the network.
© Copyright IBM Corp. 2005, 2012
47
Other antivirus software
You can use custom settings for antivirus compliance to configure the agent to detect other third-party
antivirus software. See the Custom Parameters Help for detailed information.
Configuring antivirus compliance
The agent can enforce the presence of antivirus software on the computer. You can control which
antivirus software the system should be running and you can define the actions the agent takes if the
computer is not using the required software.
If you want to block network access
The agent blocks outbound connections to the network based on the IP address lists that are set in Group
Properties in SiteProtector. You must define at least one IP address in SiteProtector before network
blocking can take place.
Important: Be sure computers can access the servers they need to update their virus software and
definitions.
Out-of-compliance settings
You can define out-of-compliance limits for computers. In addition, you can set the agent to block or limit
the computer's network access when the computer is out of compliance.
Compliance checking
You can control how frequently the agent checks the computer for compliance. You can define different
frequencies depending on whether the computer is in compliance or out of compliance. In most cases, the
agent should check the computer more frequently when the computer is out of compliance. Then, when
the user takes appropriate steps to bring the computer back into compliance, the agent detects the update
more quickly and drops the restrictions.
Navigation
Locate the policy you want to edit and then click Application Compliance > Antivirus Compliance.
48
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Advanced antivirus compliance configuration
The IBM Security Host Protection for Windows agent supports a set of custom parameters that you can
use to fine-tune antivirus compliance.
Each parameter consists of the following elements:
v Parameter name
v Expected value
v Description
Custom setup
The IBM Security Host Protection for Windows agent is designed to automatically find the specified
McAfee or Symantec antivirus software on the system. However, you can specify alternative methods of
finding the information if the initial search is unsuccessful. If the agent does not find the expected
antivirus software in the expected location, it checks for custom parameters:
v executable_name
The name of the program file that runs the antivirus scan program.
v virus_definition_disk_folder
The absolute path name to the folder where the virus definition files reside.
v virus_definition_file_name
The name of the virus definition file. Wildcard characters are accepted.
Note: If you specify a value for virus_definition_disk_folder, you should also specify a value for
virus_definition_file_name, and vice versa.
Navigation
Locate the policy you want to edit and then click Application Compliance > Antivirus Compliance.
Reference
See the Custom Parameters Help for detailed information about antivirus compliance parameters.
Chapter 6. Enforcing antivirus compliance
49
50
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 7. Enforcing application control
Intruders can place unauthorized code on a system to gain control of the system or to exploit network
vulnerabilities. When you enable application control, the agent can prevent unauthorized or modified
applications from running or connecting to a network.
Topics
“Key concepts for application control”
“How to use application control” on page 52
“How to handle unknown applications” on page 53
“How to handle known applications” on page 54
“Preparing and importing known applications” on page 56
Key concepts for application control
Intruders can place unauthorized code on a system to gain control of the system or to exploit network
vulnerabilities. When you enable application control, the agent can prevent unauthorized or modified
applications from running or connecting to a network.
Note: Application control is not currently supported on any 64-bit system or on Windows Server 2008
32-bit systems.
Level of configuration
The level of configuration required for application control varies. Before you implement your application
control strategy, you might want to monitor which applications run or connect to a network for a period
of time so you can plan the best application control strategy. This approach can help you determine the
applications that the system uses so that you can create a list of known application. You can also create or
import a list of known applications, but this requires more effort.
© Copyright IBM Corp. 2005, 2012
51
How to use application control
Application Control can prevent unauthorized applications from accessing the network. You can use
Application Control to enforce communication control for computers. This approach provides protection
from spyware, riskware, and peer-to-peer programs.
How?
To use this feature for communication control, you must create a list of authorized programs that should
be allowed to access the network (using wildcard characters and system variables, as necessary). Then,
block all other applications from accessing the network, or use the learning mode to see what other
programs are trying to connect.
Reference: “How to handle known applications” on page 54.
Preventing specific programs from running
Application Control can help enforce your corporate security policy. You can create a list of programs that
are not allowed to run. You can block known peer-to-peer programs and any other programs that users
should not be allowed to run.
How?
Create a known application rule that does not allow the program to run or access the network. If a user
attempts to run a banned application, the agent sends an event to SiteProtector.
Reference: “How to handle known applications” on page 54.
Monitoring for unapproved applications not in the corporate baseline
Before you set the policy to prevent all unknown applications from running in a production environment,
consider having the agent send alerts each time an unknown application tries to run. You can review the
alerts to see if you might need to run more applications than the baseline would allow. After you have
collected data on the types of applications that are running, you can adjust policies as needed.
Important: In a secure, locked-down environment, this approach may not be an option.
Reference: “How to handle unknown applications” on page 53.
Preventing all unapproved applications from running
If your corporate environment requires computers to be locked down, you may want to block all
unapproved applications so they cannot run.
Reference: “How to handle unknown applications” on page 53.
Navigation
Locate the policy you want to edit and then click Application Compliance > Application Control.
52
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
How to handle unknown applications
Use the Application Control settings to configure the way the agent handles unknown applications.
Important: Any application not on the known application list is considered unknown. Selecting “don't let
it connect” or “terminate” may impact the operation of the computer as the agent may prevent needed
applications from running. Running an agent with a blank or incomplete known application list with the
“terminate”” option selected could even prevent the computer from being able to start or run.
Startup parameters for unknown applications
You have several options for handling an unknown or modified application that tries to run. These
options are as follows:
v Let it run
v Let it run, but send an event to SiteProtector
v Terminate it
Access parameters for unknown applications
You have several options for handling an unknown or modified application that tries to connect to a
network. These options are as follows:
v
v
v
v
v
Let it connect
Let it connect, but send an alert to SiteProtector
Prompt user before allowing application to connect
Don't let it connect
Terminate it
Learning mode
It is not always possible to include all possible approved applications to the known applications list. The
IBM Security Host Protection for Windows agent can operate in learning mode to allow you to refine
your known applications list. In learning mode, the agent allows an unknown application to connect to a
network, but that agent also sends an alert to notify you of the activity. You can use the alert to
determine whether the application should be added to the known applications list.
Navigation
Locate the policy you want to edit and then click Application Compliance > Application Control.
Chapter 7. Enforcing application control
53
How to handle known applications
You can manage an application's ability to run or access a network on an application-by-application basis.
You have several options for handling a known application.
These options are as follows:
v Allow application to run
v Allow network access
v Block network access
v Do not allow the application to run
Important: Any application that you have not added to the known applications list is considered
unknown. If a known application is modified, it is no longer recognized and becomes unknown to the
agent.
Adding known applications
You can add known applications in the following ways:
v Manually enter the application information and settings in the user interface
v Import a list of applications and settings from a file
Note: If the agent prompts the user, the user can select Add my application to the known application
list to add the application.
Field descriptions for known applications
The known application list can use any of the following information types to identify an application and
control its behavior:
Field
Description
Rule Name
Identifies the application rule you are creating.
Description
A description of the application rule.
Path
The directory path and file name of the program file. You
can use wildcard characters and system variables.
Examples:
C:\WINDOWS\system32\program.exe
%systemroot%\system32\program.exe
%systemroot%\*.exe
Note: Application control monitors only binary program
files.
MD5
A unique set of numbers and letters that identifies a
program and version. The agent uses this value to
determine if an application has been modified.
Allow application to run
Controls whether an application is allowed to run.
Allow network access
Controls whether an application is allowed to connect to
the network.
Note: Known application entries use AND logic; the more fields you define, the more specific the entry.
For example, if you enter values for three fields, all of those values must match before an application is
considered a match.
54
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Wildcard characters
The following table lists wildcard characters and values you can use as part of a file path for known
applications:
Wildcard Character
Function
*
Matches zero or more characters.
Example: *.exe matches any file with the .exe extension, regardless of the file name.
?
Matches exactly one character.
Example: b?ss matches bass, bess, boss, b7ss.
[]
Used to designate a set of values. Matches exactly one character in the set specified within
the bracket [ ] pair unless brackets include ! or ^ characters.
Example: [r5.H_] matches r, 5, ., H, or _.
!
Used only as the first character within bracket [ ] pair. It indicates that the characters within
the bracket pair should be excluded. This character can be used interchangeably with ^.
Example: [!a3B6] matches any character except a, 3, B, 6.
^
Used only as the first character within bracket [ ] pair. It indicates that the characters within
the bracket pair should be excluded. This character can be used interchangeably with !.
Example: [^a3B6] matches any character except a, 3, B, 6.
-
Used within bracket [ ] pair to indicate a range of numbers or letters.
Example: [1-6a-d] matches any number 1 through 6 and any letter a through d.
System variables
You can use system variables as part of the file path for known applications. The following table lists the
accepted system variables and their values:
System Variable
Value
%ProgramFiles%
Program Files directory
%systemroot%
computer's root directory
%systemdrive%
drive letter for the computer's system drive
%windir%
Windows installation directory
%temp%
Windows temporary directory
%tmp%
Windows temporary directory
Including application subcomponents
You can choose to automatically include application subcomponents for known applications. Some
applications require underlying programs to run and operate correctly. Automatically including the
subcomponents of a program is a convenient option, but it is less specific and could be less secure.
If you choose not to include subcomponents, you must include the underlying programs as part of the
known application list in order for applications to function correctly.
Navigation
Locate the policy you want to edit and then click Application Compliance > Application Control.
Chapter 7. Enforcing application control
55
Preparing and importing known applications
One of the learning mode options requires the agent to send an alert to the central administrator. These
alerts show up as events in the SiteProtector Console. You can use SiteProtector to export these events
and the corresponding application information to a CSV file. Then, you can use the CSV file to import
known applications instead of adding them manually.
Handling embedded commas, spaces, or quotes
After you have exported and formatted the CSV file, use a text editor to make any additional changes.
The import functionality for known applications can handle embedded commas and quotes if you use the
proper syntax. The following rules apply:
v Data members that contain commas, spaces, or double quotes must be enclosed by a single set of
double quotes.
v Any embedded double quote must be preceded by a double quote. (The first double quote tells the
program that the next character it encounters has a special meaning.)
The following example illustrates how parsing works:
Kazaa Block,“Block all ‘“unauthorized”’ network access for Kazaa, but allow it to run”, C:\Program
Files\Kazaa\Kazaa.exe, fc3c926442cc85a469268651bd04c186,true,false
This statement is parsed as follows:
Element
Description
Rule Name
Kazaa Block
Description
Block all “unauthorized” network access for Kazaa, but
allow it to run
Path
C:\Program Files\Kazaa\Kazaa.exe
MD5
fc3c926442cc85a469268651bd04c186
Allow To Run
true
Allow Network Access
false
56
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Exporting event details from SiteProtector
Procedure
1.
2.
3.
4.
In the Enterprise Groups pane, select a group for which you want to view events.
Select the Analysis view.
Select the events you want to export.
From the Action menu, select Data Export > Export.
5. In the Files of type box, select CSV File.
6. Choose a file name and a location, and then save your CSV file.
Editing the file
Before you begin
See the SiteProtector help.
Procedure
1. Open the CSV file using a spreadsheet tool, such as Microsoft Excel.
2. Delete all columns except Path and MD5.
3. Add the following columns before the Path and MD5 columns:
v Rule Name
v Description
4. Add the following columns after the MD5 column:
v Allow to run
v Allow to access network
When you are done, the columns should be ordered as follows:
Rule Name
Description
Path
MD5
Allow to run
Allow to access
network
5. Add the appropriate data for each entry in the table, leaving blank any item you do not want to
specify.
6. Delete the row that contains the column headers and any blank rows before the first row of data.
7. Save, and then close the file.
Chapter 7. Enforcing application control
57
Importing known applications
Before you begin
Make sure that you have a CSV file prepared from SiteProtector.
Procedure
1. Select the appropriate group, and then select Application Compliance > Application Control. The
application control settings appear in the details pane.
2. Select Enable general application compliance.
3. Click Import. A browse box appears.
4. Locate the CSV file.
5. Select the file, and then click Open. The information in the file is imported into the list of known
applications.
58
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 8. Auditing system integrity and policy compliance
Suspicious activity in system log files, system registry keys, or system directories may indicate a threat to
the integrity of the host system or a violation of your security policy. The IBM Security Host Protection
for Windows agent can detect unauthorized changes to the system before they can impact availability.
Topics
“Auditing operating system logs”
“Auditing registry entries” on page 61
“Auditing files and directories” on page 65
“Auditing text logs” on page 70
Auditing operating system logs
IBM Security Host Protection for Windows can audit the Windows Event Log for suspicious activity that
may indicate a threat to the integrity of the host system or a violation of your security policy.
About event logs
The Windows event logs gather the following types of information:
v Security-related events, such as information about valid and invalid logon attempts
v Application-related events, such as an error opening an application file
v Operating system-related events, such as the failure of a system component to load
Note: You can view these event logs in the Windows Event Viewer (Administrative Tools > Event
Viewer).
© Copyright IBM Corp. 2005, 2012
59
Auditing Windows event logs
The Windows event logs can hold important clues to problems you may be having with your computer
or the applications that run on your computer. Auditing the event logs can help you identify, diagnose,
and minimize potential threats to system integrity.
IBM Security Host Protection for Windows can audit the Windows Event Log for suspicious activity that
may indicate a threat to the integrity of the host system or a violation of your security policy.
With the IBM Security Host Protection for Windows agent you can do the following:
v Configure rules that audit the event logs for frequently encountered events that may indicate your
system is under threat
v Configure custom rules that audit the event logs for specific threats to your system
About event logs
The Windows event logs gather the following types of information:
v Security-related events, such as information about valid and invalid logon attempts
v Application-related events, such as an error opening an application file
v Operating system-related events, such as the failure of a system component to load
Note: You can view these event logs in the Windows Event Viewer (Administrative Tools > Event
Viewer).
How it works
The agent monitors the event logs for events that you have specified as events of interest. When it detects
one of these events, the agent sends an alert to the Console to notify you of the log activity.
Enforcing the policy rules
Before the agent can monitor for the audit events you specify in the policy, you must ensure that the
computer is configured correctly. The enforce audit policy setting ensures that the computer has the
correct auditing configuration by integrating the agent audit alert system with the operating system audit
subsystem. When a change is detected to the auditing configuration on the operating system, the agent
ensures that the auditing configuration defined by the audit policy is enforced and combines any
additional changes detected.
Navigation
Locate the policy you want to edit and then click System Integrity Monitoring > Audit Rules.
Locate the policy you want to edit and then click System Integrity Monitoring > User Defined Audit
Rules.
60
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Creating exceptions to audit rules
As you configure protection, you create rules that define how agents should protect host systems. You
may be interested in a particular event unless the event meets a specific criteria.
How it works
You enable an audit rule to monitor and add an exception to that rule. You then edit the exception and
specify pre-defined fields in an event that you want exception to be checked. If the field that you want an
exception to be checked on is not pre-defined you can then specify a custom exception expression. Using
custom exception expressions requires that you understand the Windows event data structure. Because a
Windows event data structure can differ from one version of the operating system to another, you might
have to add multiple exceptions.
Example
You want to audit all logons to the system except logons by administrators. Using exceptions, you can
create a rule to audit the event log and then create exceptions to the rule for any administrators that are
allowed to access the system.
Navigation
Locate the policy you want to edit and then click System Integrity Monitoring > Audit Rules.
Locate the policy you want to edit and then click System Integrity Monitoring > User Defined Audit
Rules.
Auditing registry entries
The registry contains highly sensitive data, such as user profiles and the ports used by the system, so it is
essential that you secure the registry. You can provide a certain level of security by setting the
appropriate access rights to subkeys and registry entries; however, you cannot completely lock down the
registry because users and programs need access to certain areas to function correctly. Because you must
allow some access to the registry, it is essential to monitor activity in the registry so that you can quickly
identify potential threats to your system.
Chapter 8. Auditing system integrity and policy compliance
61
Monitoring the registry
Because you cannot completely lock down the registry while still allowing users and programs to access
the areas they need to function correctly, it is essential to monitor activity in the registry so that you can
quickly identify potential threats to your system.
The registry contains highly sensitive data, such as user profiles and the ports used by the system, so it is
essential that you secure the registry. You can provide a certain level of security by setting the
appropriate access rights to subkeys and registry entries; however, you cannot completely lock down the
registry because users and programs need access to certain areas to function correctly. Because you must
allow some access to the registry, it is essential to monitor activity in the registry so that you can quickly
identify potential threats to your system.
What to monitor for
The registry integrity monitoring feature can monitor the registry for events that include the following:
v Creation of subkeys (this feature is valid for version 2.0 only)
v Attempts to determine who owns a key
v Creation or modification of the value of a key
v Deletion of keys
How registry integrity works
When you monitor registry entries, you define the activity that is of interest to you for the each key. The
agent monitors the registry keys for the activity you specify and notifies you if that activity occurs.
Importing a list of keys to monitor
If you already have a list of keys the agent should monitor, you can import this list so that you do not
have to enter each key individually. You can only import content from Registration Entries files (files with
a .reg extension).
Note: The system may take several minutes to import a file. Depending on the size, it could take up to
an hour to import the list.
Navigation
Locate the policy you want to edit and then click System Integrity Monitoring > Registry Event Rules.
62
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Creating exceptions to registry monitoring
As you configure registry integrity monitoring, you create rules that define how agents should protect
host systems. You may be interested in a particular event unless the event meets a specific criteria. Using
exceptions, you can create general rules to audit the registry and specific exception rules to ignore an
event if specific criteria are met.
Example
You want to audit registry entries for successful queries unless the query was initiated by a security
manager. You create a registry integrity rule to monitor the registry entries of interest and then you create
an exception rule to exclude queries initiated by the security manager that you do not want to monitor.
Navigation
Locate the policy you want to edit and then click System Integrity Monitoring > Registry Event Rules.
Coalescing events
The IBM Security Host Protection for Windows agent, by default, generates one alert every time an event
you monitor for occurs on a system. This means that the number of alerts sent to the Console can be
significant. The agent can reduce the number of alerts sent to the console by coalescing (combining)
several instances of a similar event within a specified time frame in to one alert. This can speed up the
analysis of alerts.
Which events can you coalesce?
The IBM Security Host Protection for Windows agent supports coalescing for events detected by the
following rules:
v Audit rules
v User-defined audit rules
v Registry event rules
How it works
When you configure an audit rule to monitor the event logs for a specific event, you can set a coalescing
threshold. If the audit rule is triggered more than one time within this threshold, the agent reports all
instances of the event in one alert. The alert specifies the details of the first occurrence of the event and
the total number of times the event occurred.
Tip: Use coalescing to notify you of many occurrences of lower severity events where timely notification
is less critical.
Tip: Study the timing of events on your system to determine the best threshold setting.
Navigation
Locate the policy you want to edit and then click System Integrity Monitoring > Audit Rules.
Locate the policy you want to edit and then click System Integrity Monitoring > User Defined Audit
Rules.
Locate the policy you want to edit and then click System Integrity Monitoring > Registry Event Rules.
Chapter 8. Auditing system integrity and policy compliance
63
Identifying a suspicious series of events
Certain threats to the integrity of your system are best identified by correlating a series of events. For
example, three failed login attempts followed shortly by a successful login may indicate that a user forgot
their password; however, this may also indicate an attempt to break the system's password.
Event correlation allows you to identify suspicious patterns of events that might signify a threat to your
system; thereby allowing you to quickly pinpoint and address potential issues.
How it works
You enable certain audit rules to monitor your Windows event log. You then create a correlation rule that
defines a relationship between certain events. If these events occur within a certain timeframe and in a
certain order (if you specified the order of the events was important), the correlation rule is satisfied and
the agent notifies you.
Note: With IBM Security Host Protection Version 2.2.2, you can correlate Windows events and Registry
events.
Rules you can correlate
The agent can correlate any combination of predefined audit rules, user-defined audit rules, and registry
event rules that you have enabled up to a maximum of five rules.
Note: A correlation rule cannot contain another correlation rule.
Alerts associated with a correlation rule
The agent sends an alert for each event it is monitoring for. In addition, the agent sends another alert
when the criteria of the correlation rule is met; the name you specify for the correlation rule is the alert
name that displays in the console.
Navigation
Locate the policy you want to edit and then click System Integrity Monitoring > Correlation Rules.
64
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Auditing files and directories
By auditing directories you can detect unplanned and unauthorized changes to files on a system. This
allows you to identify and troubleshoot changes that may pose a threat to the system.
Monitoring directories and files
The file integrity monitoring component automates the detection of unplanned and unauthorized changes
to files. The agent can monitor certain system files by default. You can specify additional files to monitor
or you can remove monitored files from this list to customize your protection.
Realtime vs non-realtime monitoring
The agent can monitor files in the following modes:
v Realtime—the agent sends an alert to the Console as soon as a monitored file is modified
Consider realtime monitoring for the following situations:
– You have critical files to monitor (the agent can notify you of changes as soon as they occur)
– It is important to know who made changes to a file (only realtime monitoring can determine the
user associated with a change)
v Non-realtime—the agent sends an alert to the Console when a scheduled comparison indicates that a
monitored file has been modified
Note: A file may have been changed days before you receive the alert, depending on how frequently
you perform scheduled comparisons.
How it works
The agent creates a baseline of attribute values for the files you want to monitor. The baseline is stored in
an embedded SQLite database called iss_fim.db. The file integrity monitoring component detects
differences between the current environment and the baseline stored in the database. In realtime mode,
the comparison happens as soon as a monitored file is modified; in non-realtime mode the comparison
happens at an interval scheduled by you.
Note: The initial baseline is of the current system. The agent cannot detect files that are already
compromised; the agent will be able to notify you only of file integrity issues from this point forward.
When is the initial baseline created?
To create the initial baseline, configure the file integrity monitoring policy to define the files and file
attributes you want the agent to monitor, and then specify a complete baseline update. When the agent
receives the policy, it creates the baseline based on your settings.
Navigation
Locate the policy you want to edit and then click File Integrity Monitoring.
Chapter 8. Auditing system integrity and policy compliance
65
File attributes the agent can monitor
Global versus local attributes
The IBM Security Host Protection for Windows agent supports auditing of both global and local
attributes.
v Global attributes—the agent monitors for changes to the specified attributes for all files and folders it is
auditing
v Local attributes—the agent monitors for changes to the specified attributes only for the specified file or
folder
Support for this granular configuration allows you to define attributes of interest for all files in one place
while also allowing you to add custom protection for certain files by defining local attributes for a
specific file.
Which attributes can the agent monitor?
The agent can monitor the following file attributes:
Attribute
Description
Add
Monitors for files added to the monitored directory or
folder
Delete
Monitors for files deleted from the monitored directory
or folder
Modification Time
Note: This option is only available in Server Protection
for Windows version 2.0 or earlier.
Time that the monitored file or directory was modified
Read
Note: In Server Protection for Windows Version 2.1 and
2.2, Proventia Desktop 10.1, and Host Protection for
Windows 2.2.2 the Read option is only supported at the
local attribute level.
Monitors for any accessing of information related to a
file or directory
Note: You do not have to open a file or directory to read
information from it; whenever the system must interpret
data related to a file or directory, such as when you open
the properties window or when you cause a pop-up
window to display information about the file, you have
initiated a read.
Modify
Monitors for the following modifications to the
monitored file or directory:
v Content Checksum Monitors for changes to the
unique identifier for a file. Note that the agent uses
the SHA1 checksum algorithm.
v File Type Monitors for changes to the file format
associated with the file
v Discretionary Access Control List (DACL) Controls
which users and groups (trustees) are allowed or
denied access to a file/folder
v File Size Monitors the size of the file
v System Access Control List (SACL) A list used to
specify which attempts to access a system object are
recorded in the security event log.
v Owner Monitors for changes in the Owner of a
file/folder. For example, if C:\Dir has "Administrator"
as the Owner and then it is modified to "someuser", an
event is triggered.
66
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Importing a list of files to monitor
If you already have a list of files the agent should monitor, you can import this list so that you do not
have to enter each file individually.
File format
You can only import content from text files (files with a .txt extension), the content must contain the full
path to the file or directory, and each item must be on a new line.
Navigation
Locate the policy you want to edit and then click File Integrity Monitoring > Inclusions.
Database maintenance schedule
Schedule a database maintenance window to specify when the agent maintains and reorganize the file
integrity monitoring database.
Why schedule a database maintenance?
The file integrity monitoring database is populated as new files or directories are included for
monitoring. However, the size of the database remains unchanged when a file or directory is removed
from monitoring or removed from the file system. Scheduling database maintenance can truncate the size
of the database.
Alerts associated with a scheduled database maintenance
An agent generates an alert at the initiation of the database maintenance. The agent also generates an
alert at the completion of the task.
Navigation
Locate the policy that you want to edit and then click File Integrity Monitoring > Database
Maintenance.
Chapter 8. Auditing system integrity and policy compliance
67
Scheduling a baseline comparison
A scheduled comparison specifies when the agent should compare the current environment to the
baseline stored in the database.
Why schedule a comparison?
When the agent monitors a file in realtime, it sends an alert as soon as the monitored file is modified;
however, when the agent monitors a file in non-realtime it can only send an alert when the scheduled
comparison reveals that the file was modified.
Alerts associated with a scheduled comparison
An agent generates an alert for each modified file it detects during the scheduled comparison, regardless
of whether that file is also being monitored in realtime. The agent will generate an alert every time the
scheduled comparison runs until you update the baseline with the modified attribute value.
Agent unavailable for scheduled comparison
If for some reason the agent is not available at the time of a scheduled comparison, the agent will run the
comparison as soon as it is available. If the file integrity monitoring policy changes while the agent is
unavailable, the agent does not perform the scheduled comparison when it becomes available, but it does
update the baseline to reflect the policy changes.
Navigation
Locate the policy you want to edit and then click File Integrity Monitoring > Scheduled Comparisons.
Updating the baseline
When you update the baseline, the agent performs a one-time, immediate baseline of the monitored
attribute values for monitored files. The agent can create either a complete baseline or an incremental
baseline.
v Incremental baseline—updates the baseline to reflect only those changes since the last baseline was
created
Tip: Use an incremental baseline update when policy changes are minor; the agent can complete an
incremental baseline in a shorter period of time.
v Complete baseline—recreates the baseline based on the new policy settings
Tip: Use a complete baseline update when policy changes are substantial.
Note: A baseline update does not initiate a database comparison; database comparisons occur only as
scheduled on the Scheduled Comparisons tab.
When to update
Update the baseline whenever you make changes to the file integrity monitoring policy. Specify an
incremental or complete baseline update as part of any changes you make to the file integrity monitoring
policy.
Example: You install new software on the host system and you want to audit the integrity of the new
files. On the Inclusions tab, you add the new files to the list of files to monitor, you then specify an
incremental baseline update on the Reestablish Baseline tab to update the baseline.
68
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Important: If the agent is configured for non-realtime monitoring, consider running a scheduled
comparison before you make policy changes and update the baseline to reflect those changes. If you do
not run a scheduled comparison before you update the baseline, be aware of the following:
v Any changes made to monitored files since the last scheduled comparison will never be reported
because, as soon as the baseline is updated, the agent can no longer report changes based on the values
in the previous baseline
v Any attribute values that have changed will be saved to the baseline, whether the changes were
legitimate or not
Notification of baselining
The agent sends an alert to the console when baselining begins and when it completes.
Navigation
Locate the policy you want to edit and then click File Integrity Monitoring > Reestablish Baseline.
Excluding directories and files from monitoring
IBM Security Host Protection for Windows allows you to configure one inclusion rule to monitor many
files and then a few exclusion rules to exclude the few files that you are not interested in monitoring.
As you configure protection, you create rules that define how agents should protect host systems.
Occasionally you need to protect several files in a directory or a folder but not all files in the directory or
folder. You can support exclusions to include only the items that you want to protect.
Navigation
Locate the policy you want to edit and then click File Integrity Monitoring > Exclusions.
Chapter 8. Auditing system integrity and policy compliance
69
Auditing text logs
With the IBM Security Host Protection for Windows Server agent, you can create text log rules that
monitor log files for suspicious activity. By auditing text logs you can identify and troubleshoot changes
that might pose a threat to the system.
Text logs
With the IBM Security Host Protection for Windows Server agent, you can create text log rules that
monitor log files for suspicious activity.
When you monitor text log files in theIBM Security Host Protection for Windows Server agent, you
define rules that specify the agent to look for string patterns in certain log files. When a pattern is
detected, an event is reported to the SiteProtector System.
Note: The Text Log Monitoring component is in the IBM Security Host Protection for Windows Server
2.2.2 agent only.
How text log rules work
When you create a text log rule, you specify the string pattern to detect in a specific log file. String
patterns contain regular expressions, which are compared with each line of the log file to determine a
match. The log file is routinely monitored by the log rule, and when a match is found, a user-defined
event that has the same name as the rule is sent to the Site Protector System.
The actual path of the log file is never specified in the log rule. The agent uses log groups to associate the
log rule with the log file.
Log groups
A log group is composed of one or more log files. You can manage log files within the system by creating
multiple rules, each with unique search patterns, that refer to the same log group.
A log group can refer to:
v A single log file, which is specified by its full path
v Multiple log files, with the full path of each one separated by the pipe (|) symbol
v Multiple log files, which are specified by filename wildcard characters such as a question mark (?) or
an asterisk (*)
The log file paths that are specified in the log group can also contain system variables, which are
evaluated when the file is opened to detect and read the changes. System variables can be specified in
Windows environment variables, which have a percent sign (%) on both sides. For example,
%ProgramFiles%.
File encoding schemes
IBM Security Host Protection for Windows supports the following log file encoding schemes:
v ANSI
v UTF-8
v UTF-16
Note: Unless the Use BOM option is selected for the encoding scheme, all files within the same group
have the same encoding scheme.
70
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Navigation
Locate the policy that you want to edit and then click Text Log Monitoring.
Data identifiers for text log events
The IBM Security Host Protection for Windows agent can use data identifiers to extract certain
information from a text log entry; that information can then be used to generate custom alert information
for a particular event.
Note: When the agent extracts the values indicated by the data identifiers, the text log record must be
correctly delimited into various fields. The default delimiting characters are: space (), TAB (\t), and the
percent sign (%). These default characters can be overridden with custom delimiters when you create log
groups.
The following table describes the data identifiers that are supported by the text log monitoring
subsystem.
Data Identifier
Description
Example
@FieldN
Used to extract one information field
in a log entry where N is the relative
number of the field in the entry.
Note: When counting the number of
the field, start from zero.
A typical log message might have the
following format:
Mar 15 10:25:30 everest
sendmail[28244]: authdes_refresh:
keyserv(1m)...
To extract the host name from this
message, you would use the
following values:
v Log File Name = Host
v Value = @Field3
@File
Used to extract the name of the log
file that recorded an event you are
monitoring for.
To extract the log name from this
message, you would use the
following values:
v Name = Log File Name
v Value = @File
@HostIP
@Group
@Host
{!}
Used to extract the IP address of the
system that is triggering the event.
Note: On systems with multiple IP
addresses all of the IP addresses are
listed.
To extract the IP address, you would
use the following values:
Used to extract the Log Group name
that is associated with the rule
v Name = Log Group
Used to extract the host name that is
associated with the rule
v Name = Host Name
Used to extract a substring from a
string. {!} is a wildcard that pulls a
substring located between two
defined text entries in the entry.
To capture the user name for a user
with a failed login attempt, use the
following values:
v Name = Host IP
v Value = @HostIP
v Value = @Group
v Value = @Host
v Name = User
v Value = User {!} failed to
Chapter 8. Auditing system integrity and policy compliance
71
Wildcards for specifying log file names
When you monitor text log files with the IBM Security Host Protection for Windows agent, you can use
wildcards to specify the log file name.
Wildcard
Specifies
*
0 or more characters
?
Any one character
72
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 9. Configuring bypass filters
Bypass filtering allows you improve system performance by allowing certain packets to bypass the
protection offered by the firewall and the security event rules.
Topic
“Bypass filtering”
Bypass filtering
You can configure agents to allow packets from certain IP addresses to bypass analysis by the firewall
and the security event rules. For example, you do not need to analyze traffic related to a system backup;
you can configure a bypass filter to avoid processing this known data and slowing down the backup
process.
How it works
When the agent processes a packet, it checks to see if there is a bypass filter set for packets associated
with this IP address or this protocol. If there is a bypass filter configured, the agent does not process any
firewall rules or security event rules against the packet.
Consideration
The more bypass filters the agent must process, the greater the impact on performance; consider
configuring no more than 32 bypass filters.
Interaction among filters
The agent supports bypass filters, event filters, and trusted address filters.
v Bypass filters allow packets from certain IP addresses to bypass analysis by the firewall and the
security event rules.
v Event filters ensure that traffic associated with useful or helpful addresses is not blocked by security
event rules.
v Trusted address filters ensure that traffic associated with useful or helpful addresses is not blocked by
security event rules.
Because there are several ways for the agent to filters traffic, it is possible to enter filters that make other
filters redundant. Though the effect on the system is similar (traffic is allowed to circumvent the
protection offered by the agent), you might not see the expected behavior if you later clear a filter. If you
are unaware that there are redundant filters configured, you might expect the agent to resume processing
certain traffic after you clear a filter; however, there might still be a filter configured that allows certain
traffic to circumvent the protection offered by the agent.
References: See “Event filtering” on page 39 and “Trusted addresses” on page 36. See “Traffic seems to
be bypassing analysis” on page 86 for troubleshooting advice.
Navigation
Locate the policy you want to edit and then click Bypass Filters.
© Copyright IBM Corp. 2005, 2012
73
74
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 10. Configuring administrative settings
The IBM Security Host Protection for Windows agent offers additional features to enhance protection and
make setting up and managing agents easier.
Topics
“Configuring management settings”
“Enabling agent protection” on page 77
“Protecting during system start-up and shutdown” on page 78
“Setting up configuration sharing” on page 79
“Configuring event caching” on page 80
Configuring management settings
The management settings control how local agents interact with SiteProtector.
Setting the heartbeat interval
IBM Security Host Protection for Windows agents communicate with SiteProtector through periodic
communications called heartbeats. Agents use heartbeats to check for updated policies. The heartbeat
interval indicates the amount of time that elapses between heartbeats.
If your environment is stable, you might consider setting this value higher, such as once or twice a day. If
you are testing policy changes, you might want to temporarily set this value to as low as 30 seconds.
Important: Heartbeat settings do not affect the transmission of event information. The agent always
sends event information to SiteProtector in real time.
Enabling SiteProtector management
Even if you want to share configuration between SiteProtector and the local console, you may want to
limit certain actions from the local console. You can disable the management tab on the local console so
that users cannot disconnect the agent from SiteProtector or change which console has configuration
priority.
Responding to traffic overload
The IBM Security Host Protection for Windows agent monitors all traffic to and from the server. During
overload conditions, traffic may be delayed as the agent tries to process the high load. You can configure
the agent to allow traffic to pass-through the agent to prevent any possible delays.
The agent uses a buffer to pass packets between user space and kernel space. For each adapter on the
system, there is one buffer. The buffer is a circular queue, so, as the agent processes packets, buffer space
is made available to subsequent packets. If the agent cannot read packets from the buffer as quickly as
they are written to the buffer, the buffer fills. In these overload situations, the agent “fails open” and
forwards packets to their destination without processing them against firewall rules or security event
rules.
© Copyright IBM Corp. 2005, 2012
75
Note: This feature applies to versions 2.1 and 2.2.
Important: Allowing traffic to bypass the protection offered by the agent may impact the integrity of
your server.
Network monitoring
You can configure the IBM Security Host Protection for Windows agent to let all network traffic pass
through without being processed against firewall rules or security events (IPS and IDS mode).
Note: This feature applies to versions 2.1 and 2.2.
Configuring Agent Manager failover settings
You can configure the IBM Security Host Protection for Windows agent to report to a secondary Agent
Manager if the primary Agent Manager becomes unavailable. Specify the backup Agent Manager from
the central SiteProtector Console.
How the backup management server works
The backup management server feature works as follows:
Stage
Description
1
The central administrator configures a backup management server with port and account
settings that are identical to the main server.
2
The central administrator defines a list of available management servers in the Group
Settings policy.
Note: The first URL on the list appears in the URL text box in the Management tab on the
agent.
3
The agent attempts to report to the first management server on the list.
4
If communication with the first management server fails, the agent attempts a connection to
the next management server on the list.
Reference: For detailed information about configuring a backup management server, see the online Help.
Agent Manager failover settings
The following table describes the failover settings:
Option
Description
Number of connection attempts to current Agent
Manager before failover
Number of times the agent should try to connect to the
primary Agent Manager before it tries to connect to a
secondary Agent Manager
Initial retry interval
Number of seconds the agent should wait after a failed
connection attempt before it tries to reconnect
The retry interval doubles with each failed attempt until
a maximum of x seconds
Maximum interval (in seconds) allowed between
connection attempts
Note: The interval doubles after each failed attempt so
that multiple agents do not send frequent, repeated
communication requests.
Attempt to reconnect to primary Agent Manager after x
seconds
Interval (in seconds) after which the agent should try to
connect to the primary Agent Manager.
Note: The interval starts following a successful
connection to a secondary Agent Manager.
76
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Advanced configuration
The IBM Security Host Protection for Windows agent supports a set of custom parameters that you can
use to override default parameters. Each parameter consists of the following elements:
v Parameter name
v Expected value
v Description
Important: Do not edit the parameters file manually. The Management section of the policy provides an
easy to use interface for adding custom parameters.
Reference: See the Custom Parameters Help for detailed information about management parameters.
Navigation
Locate the policy you want to edit and then click Administration > Management.
Enabling agent protection
The IBM Security Host Protection for Windows agent can protect its own directories, services, processes,
and registry keys from tampering.
Note: If this feature was enabled in 2.0 or 2.1 it will be disabled when upgrading. Agent protection is
disabled by default, to enable for 2.1 or 2.2 agents, see the Help.
The following list details the agent protection implementation:
v 32-bit versions of Windows Server 2000 and Windows Server 2003 agent protection restricts all users on
the system, including administrators, from accessing agent directories, services, processes and registry
keys
v 32-bit version of Windows Server 2008 and Windows Server 2008 R2 for users with administrative
rights, agent protection does not provide process-level protection
v 64-bit versions of Windows Server 2003, Windows Server 2008, and Windows Server 2008 R2 for users
with administrative rights, agent protection does not provide process-level protection
Users with administrator permissions
The “Stop Agent” menu option is always available in the local console for users who have administrator
permissions for that server, regardless of the agent protection settings. If you enable agent protection and
set a password, the user must enter the password before he or she can stop the agent.
Important: When agent protection is enabled, you can only stop the agent only from the local user
interface. You cannot stop the agent from the Windows control panel or by using a command prompt. If
the local user interface is not installed, you cannot stop protected services.
Other users
The “Stop Agent” option is always disabled for users without administrator permissions for that server,
regardless of agent protection settings.
Chapter 10. Configuring administrative settings
77
Best practice
Consider requiring a password. The password option works with the other agent protection options to
secure the agent. If you set a password, only a person who knows the password can stop agent services
from the local server.
If you set a password, even users with administrator permissions must know the password to bypass
agent protection settings.
Navigation
Locate the policy you want to edit and then click Administration > Agent Protection.
Protecting during system start-up and shutdown
Some protection products leave a computer vulnerable to attack between system startup and the time the
protection utility starts. The IBM Security Host Protection for Windows agent can protect a computer
during system startup, during system shutdown, or any time that IBM Security Host Protection for
Windows protection is disabled for any reason.
How it works
The agent computer loads the driver at system startup, even before TCP is loaded. If the “Block all
network traffic when IBM Security Host Protection for Windows is not running” option is enabled, the
driver blocks network traffic until the agent is active. When agent services are disabled, the agent applies
a special firewall to temporarily allow the following UDP traffic:
v DHCP (67, 68)
v NETBIOS neighborhood (137, 138)
Deciding whether to use this option
If there are problems with the agent or protection is stopped, the agent blocks inbound traffic and
prevents the computer from accessing the network. Consider the implications for your corporate
environment before you select this feature.
Navigation
Locate the policy you want to edit and then click Administration > Agent Protection.
78
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Setting up configuration sharing
When you install an agent on a computer, you can specify whether the agent is visible to a user on the
system. You can configure agents to allow no local control, partial local control, or almost complete local
control. By default, SiteProtector has total control over all agents in the group.
Local user interface
The local interface shows events, intruders, and network traffic information on the local computer. The
interface also provides tools that allow the local user to manage security settings and to control how the
agent protects the computer.
Silent agents
An agent that does not include the local user interface is called a silent or headless agent; these agents are
always controlled from SiteProtector. Silent agents are appropriate for environments that need host-based
protection without relying on local input.
Sharing control
An agent that includes the local user interface may offer the local user a degree of control over agent
settings as follows:
Control Level
Result
Total management control
SiteProtector has complete control over the agent.
Shared management control
The local user has partial control over configuration
settings, and can alter any parameters that you have not
explicitly set. Settings configured from SiteProtector
override settings created locally.
Shared local control
The local user has control over all configuration settings.
Although SiteProtector distributes configuration settings
to all agents in the group, the local user can override any
of those settings.
Changing from headless to local console (or vice versa)
You can accomplish most configuration sharing changes by changing the configuration section of the
policy. However, you cannot change to or from a headless agent with only a policy update. If you want
to change from a headless agent to an agent with a local console or vice versa, you must prepare a new
agent build.
Navigation
Locate the policy you want to edit and then click Administration > Shared Configuration.
Chapter 10. Configuring administrative settings
79
Configuring event caching
Event caching controls whether local agents store a record of events when the computer is not connected
to SiteProtector.
How it works
If you enable this feature, the agent saves event alerts when the computer is not connected to
SiteProtector. When the agent reconnects to SiteProtector, the agent sends saved alerts to SiteProtector. If
you choose to have event alerts cached, you can set the cache size.
If you clear this option, the agent does not save event alerts when the computer is not connected to
SiteProtector and alerts for events that occur during this time are not sent to SiteProtector.
Cache size
If you choose to have event alerts cached, set the cache size (in megabytes). If the cache fills up, the agent
overwrites older alerts with newer alerts; set the cache size to minimize the loss of alerts.
Navigation
Locate the policy you want to edit and then click Administration > Event Caching.
80
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Chapter 11. Troubleshooting and support
This section describes troubleshooting issues you might encounter as you use IBM Security Host
Protection for Windows agents and lists Support contact information.
Topics
“Contacting IBM Support”
“Agent build package cannot be installed due to missing policies” on page 82
“Reporting a false positive” on page 82
“How can I restore custom policy settings for a child group?” on page 84
“Overriding parent policies does not give child policies with the parent policy” on page 84
“Under heavy traffic conditions traffic seems to be bypassing analysis” on page 85
“Not seeing any file integrity monitoring alerts” on page 85
“Traffic seems to be bypassing analysis” on page 86
“Refresh agent feature in the SiteProtector System not functioning” on page 87
“Agent showing as offline” on page 87
Contacting IBM Support
IBM Support provides assistance with product defects, answers FAQs, and helps users resolve problems
with the product.
Before you begin
Before you contact IBM Support, search for an answer or a solution by using other options first:
v See the Support portfolio topic in the Software Support Handbook for information about the types of
available support.
v Check IBM Technotes, accessible through the IBM Support Portal.
If you are unable to find an answer or a solution in the Support portfolio or in the IBM Technotes, check
to be sure your company or organization has an active IBM maintenance contract, and that you are
authorized to submit a problem to IBM, before you contact IBM Support.
Procedure
To contact IBM Support:
1. Define the problem, gather background information, and determine the severity of the problem. For
more information, see the Getting IBM support topic in the Software Support Handbook.
2. Gather diagnostic information.
3. Submit the problem to IBM Support in one of the following ways:
v By using IBM Support Assistant (ISA), if the Service Request tool is enabled on your product.
© Copyright IBM Corp. 2005, 2012
81
– Any data that has been collected can be attached to the service request. Using ISA in this way
can expedite the analysis and reduce the time to resolution.
v Online through the IBM Support Portal: You can open, update, and view all of your service
requests from the Service Request portlet on the Service Request page.
v By telephone for critical, system down, or severity 1 issues. For the telephone number to call in
your region, see the Directory of worldwide contacts web page.
Results
If the problem that you submit is for a software defect or is about missing or inaccurate documentation,
IBM Support creates an Authorized Program Analysis Report (APAR). The APAR describes the problem
in detail. Whenever possible, IBM Support provides a workaround that you can implement until the
APAR is resolved and a solution is delivered to you. IBM publishes resolved APARs on the IBM Support
website daily, so that other users who experience the same problem can benefit from the same resolution.
Agent build package cannot be installed due to missing policies
Problem
The agent installation package that is generated by SiteProtector failed to install. The installation log in
the Windows Document folder shows that the installation was not successful because policies are
missing.
Solution
To ensure that all policies are available in the agent build, complete the following steps:
1. Close and reopen the SiteProtector console.
2. Go to the policy view and make sure that all policies are correctly deployed.
3. Generate a new agent installation package from SiteProtector and use the new agent package to install
IBM Security Host Protection for Windows.
Reporting a false positive
Problem
You find that a IBM Security Host Protection for Windows agent is falsely identifying traffic, events, or
applications as threats to your system.
Background
Sometimes false positives are a result of a misconfiguration of the agent; however, sometimes, because it
is difficult to reproduce all possible network configurations when IBM Security tests its software, an
agent reports behavior as malicious or suspicious when it is not.
Solution
Report the issue to IBM Security Solutions Technical Support. Please reference the IBM Software Support
handbook for information on "Getting IBM Support."
v A screen capture of the false positive event (or events)
v A brief summary of why you think this false positive happens
v A description of the software and version information or network configuration if the false positive is
being triggered by a specific software product or network configuration in your environment
82
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Frequently, the following information is absolutely necessary for IBM Security Solutions Technical
Support to fix the false positive problem. If you can provide the following information in your report, it
would be extremely helpful:
v A capture file containing a frame by frame record of network traffic over a specific period of time.
v Explicit instructions on how to reproduce the false positive
v The name, phone, and email address of someone we can contact if we need assistance to reproduce the
false positive
Not seeing any BOEP alerts
Problem
You have enabled Buffer Overflow Exploit Protection but you are not seeing any BOEP alerts.
Background
By default, Data Execution prevention (DEP) is enabled on Windows 2003 Server and Windows XP. DEP
might block certain buffer overflow exploits before the BOEP module of the IBM Security Host Protection
for Windows agent can analyze them and send an alert.
Solution
Disable DEP if you want the IBM Security Host Protection for Windows agent to monitor for BOEP
events.
Is Version 2.0 supported on ISA Server?
Problem
You want to install IBM Security Host Protection for Windows, Version 2.0 on ISA Server but you are not
sure if this environment is supported.
Solution
IBM Security Host Protection for Windows will run in Intrusion Detection mode on ISA Servers after you
manually configure a registry key.
Configuring the agent to run on ISA Server
To
1.
2.
3.
4.
configure the agent to run on ISA Server, complete the following steps:
Install the agent.
Stop the agent from the local console.
From the command line, stop the driver using net stop issnet.
Set the registry value for HKLM/System/CurrentControlSet/Services/issnet/Parameters/
ControlFlags to 0x80000.
5. Start the agent from the local console.
Chapter 11. Troubleshooting and support
83
How can I restore custom policy settings for a child group?
Problem
You want to temporarily override child group policies and use settings from the parent group policies but
then you want to restore the custom child group settings.
Example
To test a new configuration, you temporarily need to have each child group send a heartbeat to
SiteProtector more frequently than usual; however, if you change the heartbeat setting in each child group
manually, you must make a note of the heartbeat setting for each child group so that you can restore the
setting after the testing completes.
Background
If a child group is using custom policies and you remove the overrides to the parent policies, the child
group uses the parent group policy settings. Reinstate the custom child policies by overriding the parent
policies from the child group.
Solution
Reestablish the custom settings for each child group by simply overriding the parent policy at each child
group— this restores the previous custom policy settings.
Overriding parent policies does not give child policies with the parent
policy
Problem
You want to create new custom policies for a child group but, when you override the parent group
policy, the child group policy has custom settings from a previous customization of the child group
policy not settings from the parent group policy.
Background
Overriding parent policies from a child group reinstates the previous custom policy of the child group (if
one exists).
Solution
To create a new custom child policy based on the current parent group policy settings, you must first
copy the parent policy and then paste it in the child group. After you paste the parent group policy into
the child group, you can customize the policy to create your new child group policy.
Creating a new custom child group policy
Complete the steps below to create a new custom child group policy:
1. Right-click the parent group and select Copy.
2. Right-click the child group and select Paste.
3. Select the newly pasted policy in the child group and configure your custom settings.
84
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Under heavy traffic conditions traffic seems to be bypassing analysis
Problem
Traffic seems to be reaching the system without being processed by the agent.
Background
The agent uses a buffer to pass packets between user space and kernel space. For each adapter on the
system, there is one buffer. The buffer is a circular queue, therefore, as the agent processes packets, buffer
space is made available to subsequent packets. If the agent cannot read packets from the buffer as quickly
as they are written to the buffer, the buffer fills. In these overload situations, the agent “fails open” and
forwards packets to their destination without processing them against firewall rules or security event
rules.
When the agent recovers from the overload condition, it resumes normal processing of packets—packets
are again processed against firewall rules and security event rules.
Solution
If the fail open behavior is not acceptable in your environment, for Server Protection for Windows
Version 2.1, Version 2.2, or Proventia Desktop 10.1, on the Management tab of the Administration policy,
clear the Allow traffic to pass through (fail open) check box. This will cause the agent to drop packets
when the capture buffer is full.
If the fail open behavior is not acceptable in your environment, for agent Version 2.0, in the Advanced
Configuration tab of the Security Events policy, set the packet.DropIfBufferFull parameter to TRUE. This
will cause the agent to drop packets when the capture buffer is full.
Not seeing any file integrity monitoring alerts
Problem
You updated your file integrity monitoring policy and now you are not seeing any alerts for file integrity
events.
Background
Before an agent can monitor for file integrity issues based on the settings you configured in the file
integrity monitoring policy, it must update entries in the baseline to reflect the files and values you want
to monitor.
Solution
Update the baseline whenever you make changes to the file integrity monitoring policy. Specify an
incremental or complete baseline update as part of any changes you make to the file integrity monitoring
policy.
Updating the baseline
To update the baseline, complete the following steps:
1. In the Navigation pane, select the group you updated the file integrity monitoring policy for, and then
select File Integrity Monitoring.
2. Click the Reestablish Baseline tab.
Chapter 11. Troubleshooting and support
85
3. Click one of the following options:
v Complete baseline—overwrites the existing baseline with a new baseline
Tip: Use this option if you made significant changes to the policy.
v Incremental baseline—updates the current baseline with only the new changes to the policy
Tip: Use this option if you made minor changes to the policy.
4. Do one of the following to apply the updated policy:
v Wait for the agents to send a heartbeat to the SiteProtector System
v Select the applicable group, and then select Action > Refresh Agent to force the agents to send a
heartbeat to the SiteProtector System immediately
Traffic seems to be bypassing analysis
Problem
Traffic seems to be reaching the system without being processed by the agent.
Background
The agent supports bypass filters, event filters, and trusted address filters.
v Bypass filters allow packets from certain IP addresses to bypass analysis by the firewall and the
security event rules.
v Event filters ensure that traffic associated with useful or helpful addresses is not blocked by security
event rules.
v Trusted address filters ensure that traffic associated with useful or helpful addresses is not blocked by
security event rules.
Scenario 1
It is possible that a bypass filter, an event filter, or a trusted address filter is allowing traffic to pass to the
system.
Scenario 2
It is possible that you disabled a bypass filter but you still have an event filter or a trusted address filter
that is preventing the security event rules from blocking malicious or suspicious traffic.
Scenario 3
It is possible that you disabled an event filter but you still have one or both of the following problems:
v A bypass filter that is preventing the firewall and the security event rules from blocking malicious or
suspicious traffic
v A trusted address filter that is preventing the security event rules from blocking malicious or
suspicious traffic.
Solution
Check for filters that might be preventing the agent from inspecting all the traffic you want inspected.
Note: Bypass filters and event filters can only be set from the SiteProtector System, where as trusted
addresses can be set from either the SiteProtector System or from the local console (if you are allowing
configuration sharing between the SiteProtector System and the local console). If, after checking polices at
86
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
the SiteProtector System level traffic is still not being processed as you expected, check the settings
configured at the local console.
Refresh agent feature in the SiteProtector System not functioning
You initiate a Refresh Agent request from the SiteProtector System to the IBM Security Host Protection for
Windows agent, but the agent does not initiate a heartbeat to the SiteProtector System.
Problem
You initiate a Refresh Agent request from the SiteProtector System to the agent, but the agent does not
initiate a heartbeat to the SiteProtector System.
Background
The Refresh Agent feature in the SiteProtector System uses an ICMP message to contact the agent. If you
have an ICMP firewall rule that includes your SiteProtector System address as a source to block, the
agent cannot detect the ICMP message and answer the request to heartbeat in to the SiteProtector System.
Solution
If you have an ICMP firewall rule that is blocking the SiteProtector System, you can do one of the
following actions:
v Redefine your firewall rules to allow ICMP traffic from the SiteProtector System to your agent
v Use only the regular agent heartbeat to communicate with the SiteProtector System
Agent showing as offline
The IBM Security Host Protection for Windows agent is showing as offline in the SiteProtector System,
but the agent is still sending alerts.
Problem
The agent is showing as offline in the SiteProtector System, but the agent is still sending alerts.
Background
The Unresponsive Agent Threshold setting specifies the number of minutes that can elapse between agent
heartbeat signals before the agent is considered unresponsive. If the Unresponsive Agent Threshold
setting is shorter than the heartbeat interval for an agent, SiteProtector shows your agent as offline when
in fact it is available but it has not sent a heartbeat within the threshold period.
For example, if your Unresponsive Agent Threshold is set to two hours (the default) and your heartbeat
interval is set to six hours, the agent status in the SiteProtector System changes to offline when two hours
have passed because the agent has not sent a heartbeat within those two hours. The agent will not send a
heartbeat for another four hours based on the heartbeat interval setting.
Solution
Do one of the following actions:
v Change the Unresponsive Agent Threshold setting so that it is longer than the heartbeat interval
v Send a Refresh Agent command to initiate a heartbeat from the agent to SiteProtector System
Chapter 11. Troubleshooting and support
87
System tray icon disappeared after upgrade
Problem
You have just upgraded from Version 1.0 to Version 2.0 using the heartbeat mechanism and now the
system tray icon is not visible on the host system.
Solution
Open the local console from the Start menu; the icon will again be visible in the system tray. This only
happens immediately following the update to 2.0.
88
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries.
Consult your local IBM representative for information on the products and services currently available in
your area. Any reference to an IBM product, program, or service is not intended to state or imply that
only that IBM product, program, or service may be used. Any functionally equivalent product, program,
or service that does not infringe any IBM intellectual property right may be used instead. However, it is
the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or
service.
IBM may have patents or pending patent applications covering subject matter described in this
document. The furnishing of this document does not grant you any license to these patents. You can send
license inquiries, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
U.S.A.
For license inquiries regarding double-byte (DBCS) information, contact the IBM Intellectual Property
Department in your country or send inquiries, in writing, to:
Intellectual Property Licensing
Legal and Intellectual Property Law
IBM Japan Ltd.
19-21, Nihonbashi-Hakozakicho, Chuo-ku
Tokyo 103-8510, Japan
The following paragraph does not apply to the United Kingdom or any other country where such
provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION
PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some
states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this
statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically
made to the information herein; these changes will be incorporated in new editions of the publication.
IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this
publication at any time without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in
any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of
the materials for this IBM product and use of those Web sites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without
incurring any obligation to you.
© Copyright IBM Corp. 2005, 2012
89
Licensees of this program who wish to have information about it for the purpose of enabling: (i) the
exchange of information between independently created programs and other programs (including this
one) and (ii) the mutual use of the information which has been exchanged, should contact:
IBM Corporation
Project Management
C55A/74KB
6303 Barfield Rd.,
Atlanta, GA 30328
U.S.A
Such information may be available, subject to appropriate terms and conditions, including in some cases,
payment of a fee.
The licensed program described in this document and all licensed material available for it are provided
by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or
any equivalent agreement between us.
All statements regarding IBM's future direction or intent are subject to change or withdrawal without
notice, and represent goals and objectives only.
Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business
Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be
trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at
“Copyright and trademark information” at Copyright and trademark information at www.ibm.com/
legal/copytrade.shtml.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or
both.
90
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
Index
A
advanced configuration
antivirus compliance 49
firewall 32
management settings 77
security events 40
agent
password protection 77
agent build
definition 22
Agent Manager
backup 76
failover settings 76
primary 76
secondary 76
update 9
Alert only (IDS mode) 35
alerts
associated with scheduled
comparison 68
caching 80
not in console 80, 83, 85
antivirus compliance
advanced configuration 49
custom parameters 49
overview 47
supported antivirus software 47
application control 52
handling known applications 54
learning mode 53
system variables 55
unknown applications 53
application rules 29
applications
adding to known list 54
audit events
coalescing 63
correlation 64
audit rules
exceptions 61
monitoring event log 60
predefined 60
user defined 60
automatic blocking 27
automatic port opening 31
buffer overflow exploit prevention
(BOEP) (continued)
false positives 43
Buffer Overflow Exploit Prevention
(BOEP)
and Data Execution Prevention
(DEP) 83
not seeing alerts 83
overview 44
simulation mode 45
bypass filter
interaction with other filters 73
number of 73
when to use 73
bypassing protection 75
C
caching alerts 80
coalescing
audit events 63
combining, coalescing 63
comma
in CSV file 56
communication control 52
control
application 52
communication 52
correlation
audit events 64
registry events 64
correlation rule
alert name 64
maximum number of rules 64
critical files
monitoring 65
CSV file
handling commas 56
handling double quotes 56
handling spaces 56
custom parameters
antivirus compliance 49
firewall 32
management settings 77
security events 40
B
D
backup Agent Manager 76
how it works 76
baseline
start and stop notification 69
updating 68
Block and alert (IPS mode) 35
BlockWhileAgentStopped option 78
BOEP (Buffer Overflow Exploit
Prevention) 43
buffer overflow exploit prevention
(BOEP)
data execution prevention (DEP) 43
Data Execution Prevention
and BOEP 83
disable 83
data identifiers 71
database maintenance 67
DEP (Data Execution Prevention)
deployment
agent build 22
before you deploy 5
prerequisites
Agent Manager update 9
create groups 10
© Copyright IBM Corp. 2005, 2012
deployment (continued)
prerequisites (continued)
database update 8
verify licenses 7
scenarios 6
deployment scenario checklists
deployment scenarios 5
double quotes
in CSV file 56
6
E
enforce audit policy 60
event filter 39
interaction with other filters 39
event log
auditing 60
events
coalescing 63
reducing number of 39
exceptions
audit rules 61
registry integrity rules 63
system integrity monitoring
policy 61
exclusions
file integrity monitoring 69
F
43
fail open 75, 85
failover settings
Agent Manager 76
false positives
reporting 82
file integrity monitoring
attributes monitored 66
creating initial baseline 65
database maintenance 67
database name 65
exclusions 69
global attributes 66
importing list of files 67
local attributes 66
non-realtime 65
not seeing alerts 85
overview 65
realtime 65
scheduling and non-realtime
monitoring 68
scheduling baseline comparison 68
update baseline 68
when to update baseline 68
files or folders
adding for file integrity
monitoring 65
filters
clearing had no effect 86
interaction among 36, 39, 73, 86
91
firewall
advanced configuration 32
automatic port opening 31
blocking 27
custom parameters 32
interaction with Microsoft firewall 25
manual port opening 32
opening port through 31
opening UDP port 31
protection levels 26
rule processing order 29
types of rules 28
Firewall policy 29, 30
application order 29
firewall rules
application order 29
conflicts 30
interaction with trusted addresses 36
processing order 29
Firewall Rules tab 29, 30
G
headless agent 79
changing to or from 79
heartbeat
guidelines for setting 75
understanding 75
I
IBM Security
support portal 81
technical support 81
troubleshooting 81
IDS mode 35
intrusion detection mode 35
intrusion prevention mode 35
IP addresses
trusting 36, 39
IPS mode 35
ISA Server 83
iss_fim.db 65
K
layered protection 1
learning mode 53
92
M
management settings
advanced configuration 77
custom parameters 77
manual blocking 27
manual port opening 32
Microsoft Windows firewall 25
N
network monitoring
76
T
O
overload condition
85
password protection for agents 77
policies
Firewall 29, 30
Security Events 34, 36
Text Log Monitoring 70
policy
apply updates immediately 3
group settings 21
inheritance 3
policy inheritance 3
parent settings not at child 84
restore custom policy settings 84
policy updates
when applied 3
port opening
automatic 31
manual 32
UDP traffic 31
primary Agent Manager 76
protection levels 26
H
L
security events
advanced configuration 40
custom parameters 40
rules to detect 35
Security Events policy 36
security events tab 36
silent agent 79
simulation mode for BOEP 45
space
in CSV file 56
system tray icon
not visible 17, 88
system variables
for Application Control 55
P
group settings policy 21
groups
adding 11
advantages of 10
known applications
accepted wildcard characters
adding 54
licenses 7
log groups 70
55
technical support, IBM Security 81
text log
data identifiers 71
text log rules 70
wildcards 72
traffic
bypassing analysis 86
troubleshooting 87
filter interaction 86
troubleshooting and support
contacting IBM Support 81
trust rules 37
trusted address filter
interaction with other filters 36
trusted IP address 36, 39
U
UDP traffic
opening ports 31
unknown applications 53
unresponsive agent threshold setting
update settings policy 21
upgrading 11
and local console settings 12, 13
before you upgrade 5
R
V
realtime monitoring
when to use 65
refresh agent 87
registry integrity
exceptions to rules 63
monitoring 61
reporting false positives 82
variables
for Application Control
verify licenses 7
W
S
schedule database maintenance
scheduled comparison
alerts associated with 68
secondary Agent Manager 76
55
67
wildcard characters
for known applications
wildcards
text log rules 72
Windows event log
auditing 60
IBM Security Host Protection: Administration Guide for Host Protection for Windows Servers
55
87