Foglight™ APM 5.9.9
Administration and Configuration Guide
Copyright© 2015 Dell Inc. All rights reserved.
This product is protected by U.S. and international copyright and intellectual property laws. Dell™, the Dell logo, Foglight,
IntelliProfile, NetVault, OpenManage, PerformaSure, PowerEdge, Tag and Follow, and Toad are trademarks of Dell Inc. in the
United States and/or other jurisdictions. "Apache HTTP Server", Apache, "Apache Tomcat" and "Tomcat" are trademarks of the
Apache Software Foundation. Google is a registered trademark of Google Inc. Android, Chrome, Google Play, and Nexus are
trademarks of Google Inc. Red Hat, JBoss, the JBoss logo, and Red Hat Enterprise Linux are registered trademarks of Red Hat,
Inc. in the U.S. and other countries. CentOS is a trademark of Red Hat, Inc. in the U.S. and other countries. Fedora and the
Infinity design logo are trademarks of Red Hat, Inc. Microsoft, .NET, Active Directory, Internet Explorer, Hyper-V, SharePoint,
Silverlight, SQL Server, Visual Basic, Windows, Windows Vista and Windows Server are either registered trademarks or
trademarks of Microsoft Corporation in the United States and/or other countries. AIX, IBM, PowerVM and WebSphere are
trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Sun, Oracle, Java,
Oracle Solaris, and WebLogic are trademarks or registered trademarks of Oracle and/or its affiliates in the United States and
other countries. SPARC is a registered trademark of SPARC International, Inc. in the United States and other countries. Products
bearing the SPARC trademarks are based on an architecture developed by Oracle Corporation. OpenLDAP is a registered
trademark of the OpenLDAP Foundation. HP is a registered trademark that belongs to Hewlett-Packard Development Company,
L.P. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. MySQL is a registered
trademark of MySQL AB in the United States, the European Union and other countries. Novell and eDirectory are registered
trademarks of Novell, Inc., in the United States and other countries. VMware, ESX, ESXi, vSphere, vCenter, vMotion, and vCloud
Director are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. Sybase is a
registered trademark of Sybase, Inc. The X Window System and UNIX are registered trademarks of The Open Group. Mozilla and
Firefox are registered trademarks of the Mozilla Foundation. "Eclipse", "Eclipse Foundation Member", "EclipseCon", "Eclipse
Summit", "Built on Eclipse", "Eclipse Ready" "Eclipse Incubation", and “Eclipse Proposals" are trademarks of Eclipse Foundation,
Inc. IOS is a registered trademark or trademark of Cisco Systems, Inc. and/or its affiliates in the United States and certain other
countries. Apple, iPad, iPhone, Mac OS, Safari, Swift, and Xcode are trademarks of Apple Inc., registered in the U.S. and other
countries. Ubuntu is a registered trademark of Canonical Ltd. Symantec and Veritas are trademarks or registered trademarks
of Symantec Corporation or its affiliates in the U.S. and other countries. OpenSUSE, SUSE, and YAST are registered trademarks
of SUSE LCC in the United States and other countries. Citrix, AppFlow, NetScaler, XenApp, and XenDesktop are trademarks of
Citrix Systems, Inc. and/or one or more of its subsidiaries, and may be registered in the United States Patent and Trademark
Office and in other countries. AlertSite and DéjàClick are either trademarks or registered trademarks of Boca Internet
Technologies, Inc. Samsung, Galaxy S, and Galaxy Note are registered trademarks of Samsung Electronics America, Inc. and/or
its related entities. MOTOROLA is a registered trademarks of Motorola Trademark Holdings, LLC. The Trademark BlackBerry Bold
is owned by Research In Motion Limited and is registered in the United States and may be pending or registered in other
countries. Dell is not endorsed, sponsored, affiliated with or otherwise authorized by Research In Motion Limited. Ixia and the
Ixia four-petal logo are registered trademarks or trademarks of Ixia. Opera, Opera Mini, and the O logo are trademarks of Opera
Software ASA. Tevron, the Tevron logo, and CitraTest are registered trademarks of Tevron, LLC. PostgreSQL is a registered
trademark of the PostgreSQL Global Development Group. MariaDB is a trademark or registered trademark of MariaDB
Corporation Ab in the European Union and United States of America and/or other countries. Vormetric is a registered trademark
of Vormetric, Inc. Intel, Itanium, Pentium, and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries.
Debian is a registered trademark of Software in the Public Interest, Inc. All other marks and names mentioned herein may be
trademarks of their respective companies.
Legend
CAUTION: A CAUTION icon indicates potential damage to hardware or loss of data if instructions are not followed.
WARNING: A WARNING icon indicates a potential for property damage, personal injury, or death.
IMPORTANT NOTE, NOTE, TIP, MOBILE, or VIDEO: An information icon indicates supporting information.
Foglight APM Administration and Configuration Guide
Updated - November 2015
Foglight APM Version - 5.9.9
Contents
Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Understanding how Foglight captures real user data . . . . . . . . . . . . . . . . . . . . . . . . . 8
Software components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Simplest monitoring solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Foglight APM Appliances and real user data . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Logging in to Foglight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Finding the Foglight APM dashboards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Registering Sniffers and Archivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Creating Capture Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Registering Sniffers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Registering Archivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15
Verifying that you are capturing data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Resolving gaps in traffic capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Adding missing ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Capturing complete user sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Monitoring secure web traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Verifying configuration changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24
Making your data more meaningful . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24
Focusing on selected web applications and web sites . . . . . . . . . . . . . . . . . . . . . . . . .25
Monitoring selected IP addresses only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Excluding subnets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Assigning user roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Next steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Configuring traffic capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Managing Sniffers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
Defining Sniffers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Testing Sniffer communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Managing Archivers . . . . . . . . . . . . . . . . . . . . . . . .
Defining Archivers . . . . . . . . . . . . . . . . . . . . . .
Testing Archiver communications . . . . . . . . . . . .
Setting database shard span and retention periods
. . . . . . . . . . . . . . . . . . . . . . .32
. . . . . . . . . . . . . . . . . . . . . . .33
. . . . . . . . . . . . . . . . . . . . . . .34
. . . . . . . . . . . . . . . . . . . . . . .34
Managing capture groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
Planning capture groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Defining capture groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Assigning Sniffers and Archivers to capture groups . . . . . . . . . . . . . . . . . . . . . . . .37
Testing capture group communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
Managing sessionizing policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
Understanding the default sessionizing policy . . . . . . . . . . . . . . . . . . . . . . . . . . .39
Deciding when to add more sessionizing policies . . . . . . . . . . . . . . . . . . . . . . . . .40
Understanding how sessionizing policies are applied . . . . . . . . . . . . . . . . . . . . . .41
Defining sessionizing policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41
Sessionizing policies—Identifying user sessions . . . . . . . . . . . . . . . . . . . . . . . . . .42
Foglight APM 5.9.9 Administration and Configuration Guide
Contents
3
Sessionizing policies—Associating user names with user sessions . . . . . . . . . . . . . . .46
Sessionizing policies—Targeting sessionizing policies to URL prefixes . . . . . . . . . . . .49
Sessionizing policies—Defining when a session has ended . . . . . . . . . . . . . . . . . . .51
Managing monitored ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52
Discovering ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52
Defining monitored ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
Managing monitored IP addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54
Discovering IP addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Defining monitored IP addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56
Displaying host names instead of IP addresses . . . . . . . . . . . . . . . . . . . . . . . . . . .57
Monitoring secure (HTTPS) web traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58
Managing monitored subnets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58
Discovering subnets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59
Defining monitored subnets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60
Managing identifiers for virtual addressing schemes . . . . . . . . . . . . . . . . . . . . . . . . .61
Defining identifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .62
Capturing originating client IP addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63
Capturing server IP addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63
Capturing host names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63
Managing private keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64
Managing local private keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64
Managing HSM private keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .66
Managing web services and SOAP operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67
Creating web services automatically from WSDL files . . . . . . . . . . . . . . . . . . . . . .68
Defining web services and SOAP operations . . . . . . . . . . . . . . . . . . . . . . . . . . . .68
Configuring traffic analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Managing custom fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Predefined custom fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Setting and updating the value of custom fields . . . . . . . . . . . . . . . . . . . .
Defining custom fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
...
...
...
...
. .70
. .71
. .71
. .71
Managing custom metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74
Editing custom metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74
Managing security policies for sensitive hit details . . . . . . . . . . . . . . . . . . . . . . . . . .75
Hiding or replacing sensitive hit details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75
Omitting sensitive hit details from the database . . . . . . . . . . . . . . . . . . . . . . . . .76
Defining sensitive hit details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76
Managing security policies for sensitive content . . . . . . . . . . . . . . . . . . . . . . . . . . . .77
Hiding or replacing sensitive content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .78
Omitting sensitive content from the database . . . . . . . . . . . . . . . . . . . . . . . . . . .79
Defining sensitive content expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79
Managing pivots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80
Understanding pivot categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80
Enabling pivot categories and defining their membership . . . . . . . . . . . . . . . . . . .82
Enabling global pivots for web sites and endpoints . . . . . . . . . . . . . . . . . . . . . . . .84
Configuring advanced options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .84
Managing domain rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85
Foglight APM 5.9.9 Administration and Configuration Guide
Contents
4
Discovering aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .86
Defining domain rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .86
Storing large hit details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87
Defining large hit details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
Managing special events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
Defining global stop criteria for sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89
Defining global stop criteria for sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90
Defining special events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90
Troubleshooting improperly classified URLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
Defining URL classifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
Managing user agent rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .93
Defining new user agent rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .93
Setting the execution order for rule files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95
Defining additional parsed response types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95
Defining captured request body types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .96
Modifying traffic analysis options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97
Managing scripts . . . . . . . . . . .
Before you begin . . . . . . . .
Determining requirements . .
Creating scripts . . . . . . . . .
Testing scripts . . . . . . . . . .
Defining scripts . . . . . . . . .
....
....
....
....
....
....
....
....
....
....
....
....
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .98
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .98
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Configuring analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .105
Introducing hit analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Reviewing default hit analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Tips for creating effective hit analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Configuring hit analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Introducing sequence analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Configuring sequence analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Introducing session analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Tips for creating effective session analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Configuring session analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Summary of options by analyzer type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Configuring conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Defining match conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Defining error and warning conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
Grouping multiple conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Defining conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Working with metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Understanding standard metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Creating custom metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Changing the name of metric topology objects . . . . . . . . . . . . . . . . . . . . . . . . . 121
Understanding pivots for analyzer metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Publishing hit metrics for use in the Geographical Perspective dashboard . . . . . . . 124
Defining service level agreement thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Updating custom fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Foglight APM 5.9.9 Administration and Configuration Guide
Contents
5
Updating the value of custom fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Creating new custom fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Defining custom field updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Executing scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Executing scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Creating new scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Defining storage policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Defining hit storage restrictions for hit analyzers . . . . . . . . . . . . . . . . . . . . . . . . 129
Defining storage options for sequence analyzers . . . . . . . . . . . . . . . . . . . . . . . . 130
Setting searchability options for analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Defining options for hit analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Defining options for session analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Defining events for sequence analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Understanding sequence states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Defining event conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
Recording the order of triggered events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
Defining sequence events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Capturing traffic outside the network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .139
Deciding when to add browser instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Introducing the instrumentation components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Instrumentation library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Instrumentation Listener . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Configuring an instrumented monitoring solution . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Verifying instrumentation payloads are received . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Viewing metrics collected by instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Configuring advanced options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .144
Configuring internal settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Archiver query settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Replay settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Metadata settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Substituting static page elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Defining static page elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Excluding subnets from data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Defining subnet filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Integrating Foglight APM with other products . . . . . . . . . . . . . . . . . . . . . . . . . . . .152
Integrating with CA SiteMinder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Adding SiteMinder Server definitions in Foglight . . . . . . . . . . . . . . . . . . . . . . . . 153
Registering Foglight as a SiteMinder agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Adding SiteMinder to a sessionizing policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Integrating with SafeNet Hardware Security Modules . . . . . . . . . . . . . . . . . . . . . . . . 155
Planning a Foglight–HSM integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Registering HSM servers in Foglight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Generating SafeNet HSM client certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Associating HSM keys with monitored IP addresses . . . . . . . . . . . . . . . . . . . . . . . 157
Foglight APM 5.9.9 Administration and Configuration Guide
Contents
6
Maintaining and troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .158
Checking the health of Foglight APM Appliances . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
Investigating issues with appliance health . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
Summary of Appliance Health views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Managing configuration changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Reviewing change versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Understanding configuration modules and objects . . . . . . . . . . . . . . . . . . . . . . . 167
Exporting configuration modules and objects . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Restoring and importing configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Reviewing warnings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Creating support bundles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Updating appliance software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Restarting the embedded Agent Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Appendix: Building regular expressions in Foglight . . . . . . . . . . . . . . . . . . . . . . . .171
What is a regular expression? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Where can I find regular expressions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Regular expression basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
Building a simple pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Building a pattern that matches a specific character . . . . . . . . . . . . . . . . . . . . . 174
Building a pattern that matches multiple characters . . . . . . . . . . . . . . . . . . . . . 174
Using advanced quantifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Using special characters and regular expression flags . . . . . . . . . . . . . . . . . . . . . 175
Grouping elements in a pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
Additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
Appendix: Traffic analysis processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .177
Understanding the traffic analysis workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Stage 1: Run hit analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
Stage 2: Run sequence analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Stage 3: Finalize stopped sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Stage 4: Finalize the session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Example of interdependent analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Contacting Dell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Technical support resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Foglight APM 5.9.9 Administration and Configuration Guide
Contents
7
1
Getting started
Foglight™ APM for Real User Experience (hereafter called Foglight APM) is an appliance-based application
performance management solution. The Administration and Configuration Guide walks you through configuring
Foglight APM to capture, decrypt, and store data about how real users navigate your monitored web sites and
web applications. This guide is intended for Foglight users with the role of APM Administrator.
As soon as your Network Administrator finishes installing and setting up Foglight APM Appliances, the Sniffers on
the appliances begin capturing and analyzing data about hits and user sessions. As an APM Administrator, your
first task is to verify that the data being collected reflects your entire monitored environment. Then you can
begin to refine your data collection.
This workflow walks you through the following steps:
1
Understanding how Foglight captures real user data
2
Logging in to Foglight
3
Finding the Foglight APM dashboards
4
Registering Sniffers and Archivers
5
Verifying that you are capturing data
6
Resolving gaps in traffic capture
7
Making your data more meaningful
8
Focusing on selected web applications and web sites
9
Assigning user roles
10 Next steps
These steps assume that you are configuring Foglight APM Appliances for the first time.
Understanding how Foglight captures
real user data
Foglight APM Appliances can capture, decode, decrypt, analyze, and store data about how real users navigate
and use your web sites and web applications. This section introduces the software components used by Foglight
APM Appliances and the data collected.
This section introduces the following concepts:
•
Software components
•
Simplest monitoring solution
•
Foglight APM Appliances and real user data
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
8
Software components
A monitoring solution includes the following components:
•
Foglight Management Server
•
Archiver
•
Sniffer
•
Browser instrumentation
•
Relayer
•
Archiver databases
Foglight Management Server
For real user monitoring, the Foglight Management Server communicates with Sniffers and Archivers, queries
the Archiver databases, stores Foglight-generated metrics, and performs some administrative tasks.
The Foglight browser interface contains dashboards for real user monitoring—organized under the APM menu—
where you can:
•
configure traffic capture options (such as capture groups and sessionizing policies)
•
configure traffic analysis options (such as applications and analyzers)
•
search for and display details and metrics associated with hits, sequences, and sessions
•
review transactions and explore sequences
•
view geographical data on a map
•
replay end-user sessions
•
create and distribute reports
•
check the health of your appliances
You can use the Foglight Administration dashboards to perform various administrative tasks, such as assigning
user roles and groups.
Archiver
An Archiver receives data from the Sniffers in its capture group. It performs traffic analysis tasks on the data,
such as running analyzers on hits, sequences, or sessions to generate additional details, fields, and metrics. The
Archiver then stores all the data (captured and generated) in its local Archiver database for searching and
reporting.
Sniffer
A Sniffer captures raw packets of web traffic crossing its tap points on the network and/or from instrumented
browser pages. Sniffers decode, decrypt, and analyze the packets. This includes activities such as SSL
decryption and reassembling hit-level and page-level fields. The Sniffer transmits the data to an Archiver within
the same capture group for further analysis and storage. A capture group defines a set of Sniffers and Archivers
that work together.
Browser instrumentation
While more invasive than the passive monitoring performed by a Sniffer, browser instrumentation is sometimes
the only reliable way to track down performance issues occurring outside of your own web servers, such as slow
content service by external web servers in a Content Delivery Network (CDN) or inefficient processing by client
browsers in a Web 2.0 application. Instrumentation has two parts: the instrumentation library and the
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
9
instrumentation listener. You add the Foglight APM instrumentation library to a web site or a web page using as
a JavaScript include statement. An onsite web server and the Sniffer together act as the instrumentation
listener. The web server receives data from the instrumented pages and the Sniffer, which is monitoring the
server, parses out the data and sends it on to the Archiver with all the other captured data.
Relayer
Each Sniffer and Archiver contains a Relayer component that handles communication from the Sniffers to the
Archivers. Relayers also function as load-balancers, distributing captured data evenly among the Archivers
within a capture group.
Archiver databases
Storage is implemented as a distributed, searchable database, intelligently partitioned and managed across
multiple Archiver databases. Each Archiver has an Archiver database on the same appliance. Archivers store
data in database shards. The size of each shard is limited to a maximum size or a maximum time span,
whichever is reached first.
An Archiver maintains its subset of the database locally, and each Archiver has a complete view of each
individual user session it receives. If an Archiver in a capture group fails, its user sessions are unavailable until
the Archiver is repaired, but the rest of the Archiver database can still be used.
The database structure is described by a set of Java™ model objects called dbmodel.
Simplest monitoring solution
A small installation may be able to use an All-in-One Appliance, where the Foglight Management Server, Sniffer,
and Archiver components all reside on the same appliance. The following diagram shows how an All-in-One
Appliance fits into a network.
Figure 1. Simplest monitoring solution
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
10
The software components can run on either Dell PowerEdge™ appliances or on virtual appliances hosted on a
VMware® vSphere® Server. Typical installations usually require multiple components hosted on different
appliances. To see diagrams representing larger monitoring solutions, see the Foglight APM Appliances
Installation and Setup Guide.
Foglight APM Appliances and real user data
Real user data includes:
•
details about hits, such as the page URL, client IP address, and request header
•
details about sessions, such as the session identifier and a record of the hits that occurred within the
session
•
Foglight-generated metrics, such as the time required to load a page, total hits on a page, and number
of hits in a sequence or session
For a complete list of the details and metrics collected, see the Foglight APM Reference Guide.
Logging in to Foglight
In a fresh installation of Foglight, the default user name and password are foglight/foglight.
TIP: In an existing installation of Foglight, if you (the APM Administrator) are not the Foglight
Administrator, contact the Foglight Administrator for your user credentials.
To log in to Foglight:
1
Open a supported browser.
2
In the navigation bar, enter the server IP address where Foglight is installed followed by :8080.
Example: http://<Foglight_IP_address>:8080
3
Log in using your Foglight user credentials.
Finding the Foglight APM dashboards
To find the Foglight APM dashboards, in the Foglight navigation panel, under Dashboards, expand APM. Top-level
APM dashboards enable you to view aggregated data by sequence, transaction, or geographical perspective. The
rest of the dashboards are organized into menu groupings that reflect configuration tasks, search activities, and
support tasks.
The APM menu contains the following menu items:
•
•
Configure—configure the software components, modify what data is captured, and define how the data
should be handled or analyzed before being stored.
•
Traffic Analysis—configure what happens to the data after it has been captured. You can define
applications, analyzers, and sensitive data policies.
•
Traffic Capture—configure the software components and assign them to capture groups. Set
options to resolve gaps in the data captured by default, decrypt secure traffic, and capture
additional information that makes the data more meaningful.
Configure Advanced Options—integrate with other products and configure less frequently used options
to better reflect your environment. Usually, you can leave the advanced options with default values until
you need to change them.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
11
•
Error Explorer—display the top sampled requests with errors for a selected transaction during a
selected timeslice.
•
Geographical Perspective—assess the performance of your web sites and applications by city,
country, or region.
•
Response Time Breakdown—display the top sampled requests for a selected transaction during a
selected timeslice.
•
Search—search the Archiver databases for details about hits, sequences, and sessions. For example, this
workflow uses the Hits dashboard to verify that Foglight is collecting data.
•
Sequence Explorer—assess the performance of important workflows in your web applications.
•
Support—monitor the performance of your appliances and manage saved configurations.
•
Transactions—monitor real user experiences with or without the other transactional data captured by
Foglight.
•
Web Site and Endpoints—assess the performance of your web sites and endpoints.
Figure 2. APM dashboards menu
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
12
This Getting Started workflow introduces you to some of the dashboards available under the Configure and
Search menus. The rest of the dashboards are covered in the other sections of this guide and in the Foglight
APM User Guide.
Registering Sniffers and Archivers
NOTE: If you are using an All-in-One Appliance with no additional appliances, you can skip this step. The
local Sniffer and Archiver are already preregistered and assigned to a default capture group.
In a multi-appliance environment, you may have one or more Sniffers and one or more Archivers hosted on
different appliances. Sniffers capture, decrypt, and analyze web traffic, and transmit content and metrics to
one or more Archivers within the same capture group as the Sniffer. Archivers capture content and metrics
collected by Sniffers, analyze that content, and maintain databases used for searching, reporting, and replay.
For more information about how Archivers work, see Understanding how Foglight captures real user data.
To see the data being captured and stored by Sniffers and Archivers, you need to register the Sniffers and
Archivers and optionally organize them into capture groups.
This section covers the following topics:
•
Creating Capture Groups
•
Registering Sniffers
•
Registering Archivers
Creating Capture Groups
By default, all registered Sniffers and Archivers are assigned to the Default Capture Group. If you want, you can
create your own capture groups to separate data coming from distinct areas of the network or different
geographical locations. For more information, see Planning capture groups.
To add a capture group:
1
Click APM > Configure > Traffic Capture > Capture Groups.
2
Click Add.
3
In the Add Capture Group dialog box, complete the options. For help with options, see Defining capture
groups.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
13
NOTE: The Sniffers and Archivers areas are display only. To add Sniffers and Archivers to the group,
see Registering Sniffers and Registering Archivers.
4
Click Save.
Registering Sniffers
When using an All-in-One Appliance, the Sniffer component comes preregistered as the Local Sniffer—you do
not need to register it. For installations with standalone Sniffer Appliances, you need to add and configure the
Sniffers.
To add a Sniffer definition:
1
Click APM > Configure > Traffic Capture > Sniffers.
2
Click Add.
3
In the Add Sniffer dialog box, complete the options. For help with options, see Defining Sniffers.
NOTE: From the Capture Group list, you can select a capture group you defined earlier. For more
information, see Creating Capture Groups.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
14
4
Click Save.
Registering Archivers
When using an All-in-One Appliance, the Archiver component comes preregistered as the Local Archiver—you do
not need to register it. For installations with standalone Archiver Appliances, you need to add and configure the
Archivers.
To add an Archiver definition:
1
Click APM > Configure > Traffic Capture > Archivers.
2
Click Add.
3
In the Add Archiver dialog box, complete the options. For help with options, see Defining Archivers.
IMPORTANT: From the Capture Group list, you can select a capture group you defined earlier. For
more information, see Creating Capture Groups.
4
Click Save.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
15
Verifying that you are capturing data
To verify that registered Sniffers are actively capturing data, perform a Search Hits search.
To verify that the default Sniffer is capturing data:
1
Click APM > Search > Hits.
2
In the Search Hits dashboard, click Search.
The Search button reruns the last search performed. In a new installation, the search returns all hits that
occurred within the last 15 minutes. You can find the search parameters beside Search Description at the
top of the dashboard.
If you see data in the table, you know that Sniffers are capturing data.
If the search ends with no data, the Sniffers’ appliances may not be installed or configured correctly.
Contact your Network Administrator.
Resolving gaps in traffic capture
The next step is to ensure that Sniffers can capture, and if necessary decrypt, the traffic for all the web
applications and web sites you want to monitor. For example, you may need to register a (non-default) port
number used by your application or provide private keys to decrypt secure web traffic.
To determine if you need to adjust your Sniffer-based settings, consider the situations and symptoms in the
following table, and then click the related link to get instructions on how to update options. When you finish
making changes, run another hit search to ensure the data gaps are resolved. For more information, see
Verifying configuration changes.
NOTE: To locate the issues described in the Symptom column, look in the Search Hits results that you
generated earlier in this workflow. For more information, see Verifying that you are capturing data.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
16
Table 1. Resolving gaps in traffic capture
Situation/Requirement
Symptom (in Search Hit results)
Instructions
Your web applications use ports
In the Hits column, you are missing
other than the default ports 80, 443, HTTP hits for some web
and 8080.
applications.
Adding missing ports
Your web applications use session
identifier variables other than the
defaults or you use single sign-on
technology.
Capturing complete user sessions
Find an HTML hit in the table and
click its Session Explorer
icon.
In the Session Explorer, the user
sessions seem incomplete or you do
not have any session-level
information.
You want to monitor secure (HTTPS) In the Hits column, you are missing
web traffic.
HTTPS hits for all web applications.
Monitoring secure web traffic
You want to monitor Web 2.0
applications, web sites/applications
hosted in the cloud, performance of
content delivery networks, or
mashup web sites and applications.
Finish all the Getting Started
topics. Request the document
“Capturing Traffic Outside the
Network” from Dell Support.
In the Hits column, you are missing
details for activities occurring
outside your network, such as
browser-level processing, external
web server activity, or linked social
network activity.
This section covers the following topics:
•
Adding missing ports
•
Capturing complete user sessions
•
Monitoring secure web traffic
•
Verifying configuration changes
Adding missing ports
By default, a Sniffer monitors ports 80, 443, and 8080. If your web applications use different port numbers, a
Sniffer cannot capture data until you include those port numbers in the traffic capture settings. You can use the
Discovery tool to generate a list of ports that are currently handling live web traffic, and then select and add
ports from the list. For more information, see Managing monitored ports.
To discover and add monitored ports:
1
Click APM > Configure > Traffic Capture > Monitored Ports.
2
Click Discover.
The Port Discovery dialog box appears, and begins displaying ports in use in the monitored environment.
You can use the Search field to narrow the results.
3
When you see the ports that your web applications use, click Stop.
4
Select the check boxes beside the ports you want to add.
NOTE: You do not need to select the default ports 80, 443, and 8080.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
17
5
Click Add.
All Sniffers begin monitoring traffic on these ports.
Capturing complete user sessions
For Foglight to understand how user sessions are identified and managed within your monitored web
applications, you need to define a sessionizing policy. Usually, you can edit the default sessionizing policy to
include your application’s session identifiers. However, if you are monitoring Microsoft® SharePoint®, Oracle®
PeopleSoft, or Oracle® Siebel, you need to create a new policy. For more information, see Deciding when to add
more sessionizing policies.
By default, Sniffers look for session identifiers under the following default variable names:
•
aspsessionid (default for ASP applications)
•
asp.net_sessionid (default for ASP .NET® applications)
•
sessionid
•
jsessionid — cookie (default for Java™ applications)
•
jsessionid — URL parameter (default for Java™ applications)
•
phpsessid (default for PHP applications)
•
siteserver.id
If your web application uses different variables to pass session identifiers, you need to add them. If you know
that the session identifiers are in cookies, you can try to discover the variable names in the live web traffic, as
described in the first procedure. Alternatively, you can ask your application developers for the variable names
and add them manually, as described in the second procedure.
NOTE: If your network uses CA SiteMinder Servers for single sign-on, you need to integrate Foglight with
your SiteMinder servers and add the SiteMinder cookie as the session identifier. For instructions, see
Integrating with CA SiteMinder.
Typically, your sessionizing policies should align closely with the session management techniques employed by
your applications. You can set options that define boundaries for sessions, such as the maximum number of hits
per session, the maximum length of a session, and timeout rules. If the default values for these options do not
match your session management values, you can change them (For more information, see Sessionizing policies—
Defining when a session has ended.)
For more information, see the following topics:
•
Adding session identifier variables in cookies
•
Adding other types of session identifier variables
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
18
•
Prioritizing session identifiers
Adding session identifier variables in cookies
The easiest way to add session identifier variables found in cookies is to use the Discovery tool. For more
information, see Sessionizing policies—Identifying user sessions.
To discover and add session identifier variables in cookies:
1
Click APM > Configure > Traffic Capture > Sessionizing Policies.
2
Expand Session Identifiers.
3
Click Discover.
The Session Identifiers Discovery dialog box appears, and begins displaying cookies found in live web
traffic. You can sort the results by any column heading or use the search box to show only a subset of
Cookies based on the cookie name.
4
When you see the cookies that you want to add, click Stop.
5
Select the check boxes beside the cookies you want to add.
6
Click Add.
The cookies appear as session identifier variables in the Session Identifiers table.
Adding other types of session identifier variables
For all other types of session identifier variables, you need to add them one at a time.
To add other types of session identifier variables:
1
Click APM > Configure > Traffic Capture > Sessionizing Policies.
2
Expand Session Identifiers.
3
Click Add.
4
In the Add Session Identifier dialog box, complete the options. For help with options, see Defining
session identifiers.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
19
5
Click Save.
The session identifier variable appears in the Session Identifiers table.
Prioritizing session identifiers
When you finish adding session identifier variables, you should prioritize the variables within the Session
Identifiers table (using the Up
and Down
icons) so that the most significant variables appear at the top of
the table. If a hit contains more than one registered session identifier, the first variable in the table that
matches one of the session identifiers (searching from top to bottom) is used to track the current session. For
more information, see Prioritizing session identifiers.
Monitoring secure web traffic
HTTPS hits are not displayed in the search results because the Sniffer does not have access to the information
necessary to decrypt your secure web traffic. To display information about HTTPS hits, you need to register
secure ports (if using a non-default port), provide the private keys used to encrypt the web traffic, and
associate the port and key with the IP address of the server hosting the web site or application. For more
information, see Monitoring secure (HTTPS) web traffic.
TIP: If you do not immediately have access to the private keys, you may want to complete Step 1 and Step
2 now, and finish this procedure when you have access to the keys.
Monitoring secure web traffic requires the following steps:
Step 1: Registering secure ports
Step 2: Specifying monitored IP addresses
Step 3: Providing private keys
Step 4: Associating ports and keys with monitored IP addresses
Step 1: Registering secure ports
Port 443 is registered by default. If you need to register other HTTPS-enabled ports, see Adding missing ports.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
20
Step 2: Specifying monitored IP addresses
You can use the Discovery tool to detect traffic on IP addresses currently in use in live web traffic. You can then
select the IP addresses that carry the secure traffic you want to monitor. If an IP address is currently inactive,
you need to add it manually. For more information, see Managing monitored IP addresses.
To discover and add IP addresses:
1
Click APM > Configure > Traffic Capture > Monitored IP Addresses.
2
Click Discover.
The IP Discovery dialog box appears and begins displaying IP addresses in use in the monitored
environment. You can use the Search field to narrow the results.
3
When you see IP addresses using the HTTPS-enabled ports you registered, click Stop.
4
Click the Ports column heading to sort IP addresses by port numbers.
5
Select the check boxes beside the HTTPS-enabled ports you registered.
6
Click Add.
Step 3: Providing private keys
Private keys can be stored locally on Foglight APM Appliances or accessed from an HSM appliance. For more
information, see Managing private keys.
IMPORTANT: If you use SafeNet HSM appliances to secure your private keys, you need to integrate Foglight
with each HSM. For more information, see Integrating with SafeNet Hardware Security Modules.
To add a local private key:
1
On the navigation panel, under Dashboards, click APM > Configure > Traffic Capture > Private Keys.
2
Click Add Local Key.
3
In the dialog box, complete the options. For help with options, see Defining local private keys.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
21
4
Click Save.
The key appears in the Private Keys table.
5
If your web server supports multiple domains from the same IP address and TCP port number, specify the
domain that requires this key.
a
In the Domains column, click the monitored IP address’ Click to add domains link.
b
In the Domains dialog box, click Add.
c
Specify the domain name, and click Save.
The domain appears in the table.
d
Repeat to add any other domains that require this key.
e
Click Save.
The dialog box closes. The domains appear in the Private Keys table.
To add an HSM private key:
1
On the navigation panel, under Dashboards, click APM > Configure > Traffic Capture > Private Keys.
2
Click Add HSM Key.
3
In the dialog box, complete the options. For help with options, see Defining HSM private keys.
4
Click Save.
The key appears in the Private Keys table.
5
If your web server supports multiple domains from the same IP address and TCP port number, specify the
domain that requires this key.
a
In the Domains column, click the monitored IP address’ Click to add domains link.
b
In the Domains dialog box, click Add.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
22
c
Specify the domain name, and click Save.
The domain appears in the table.
d
Repeat to add any other domains that require this key.
e
Click Save.
The dialog box closes. The domains appear in the Private Keys table.
6
Repeat to add other private keys.
Step 4: Associating ports and keys with monitored IP
addresses
A monitored IP address can have one or more keys associated with it. If your web server is configured to support
multiple domains from the same IP address and TCP port number (using Server Name Identification (SNI)), you
also need to select the domain when you select the key. For more information, see Monitoring secure web
traffic.
TIP: If the private key you need is not available in the Add Private Key dialog box, you need to add the key
on the Private Keys dashboard. For more information, see Step 3: Providing private keys. If a port is not
available, see Adding missing ports.
To associate private keys with an IP address:
1
Click APM > Configure > Traffic Capture > Monitored IP Addresses.
2
In the row containing the IP address, click Private Keys
.
The Monitored IP Addresses > Private Keys view appears. The table contains a list of private keys (if any)
associated with the monitored IP address.
3
Click Add.
The Add Private Key dialog box appears.
a
From the Private Key list, select a key.
The list contains private keys that were previously defined on the Private Keys dashboard.
b
Select the port that carries web traffic encrypted using this private key.
The list contains monitored ports configured for HTTPS that were previously defined on the
Monitored Ports dashboard.
c
Optional—if the private key is valid for a limited time, you can include the time frame so that you
know when to request an updated key. Select Enforce Key Usage Time Frame, and specify the
start date/time and the end date/time.
d
Click Save.
The private key appears in the table and is now associated with the IP address.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
23
4
Repeat to add other private keys required by the device located at this IP address.
Verifying configuration changes
After you make your changes, run another default Search Hits search. If you are still missing hits for one of your
applications, the application may not be actively handling web traffic. Use the application (log in or perform a
task) to generate a hit and then run another Search Hits search. You may need to sort the results table or use
the search box to locate the hit.
TIP: The default search shows results for the last 15 minutes, so unless you wait 15 minutes after making
a change to your configuration (or define a Simple Search with a shorter time range), you may continue to
see some previously collected data in your new search results. By default, search results are sorted by the
Time column, so hits captured after changes to your traffic capture configuration should appear at the
top of the table.
If the hit you generated is still missing from your results, it may be that traffic for the application is not crossing
one of the Sniffers’ tap points on the network. Contact your Network Administrator to discuss your options.
Making your data more meaningful
After you confirm that Sniffers are able to capture hit and session details for all the web traffic you want to
monitor, you can evaluate the quality and completeness of your data. You may find areas where you can
improve the quality of the data collected. You may also want to annotate the data with additional information
to support searching and reporting tasks. Many customers choose to do some of these tasks now, during the
initial configuration, and postpone other tasks until after they become familiar with the data collected.
Consider the situations/requirements and symptoms in the following table. If you want to make changes, click
the link in the Instructions column to go to the relevant sections within this guide.
NOTE: To locate the issues described in the Symptom column, in most cases you can start from the Search
Hits results that you generated earlier in this workflow. For more information, see Verifying that you are
capturing data.
Table 2. Making data more meaningful
Situation/Requirement
Symptom (in Search Hits results)
Instructions
The network uses a proxy
server to route client traffic.
In the Client IP column, most of the hits
are coming from the same client IP
address.
Capturing originating client IP
addresses
The network uses a load
balancer, reverse proxy, or
virtual IP addressing scheme.
In the Server IP column, most of the
server IP addresses are the same.
Capturing server IP addresses
Identify user sessions by the
user name.
Click Session Explorer
beside any
HTML hit. At the top of the Session
Explorer, the User Name field is blank.
Sessionizing policies—Associating
user names with user sessions
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
24
Table 2. Making data more meaningful
Situation/Requirement
Symptom (in Search Hits results)
Instructions
Map SOAP operations to web
services.
First visit the web service to generate
some hits. In the Search Hits dashboard,
run a Simple Search and select the
content category SOAP. In the search
results, click Hit Detail View beside a
SOAP hit. In the Hit Detail view, under
Request Content, the SOAP Web Service
Name field is blank.
Managing web services and SOAP
operations
Annotate data with the subnet
(for intranet applications).
None. You need to specify your subnets.
Managing monitored subnets
Search for data using the name In the Simple Search and Expert Search
dialog boxes, the Analyzer list is empty
of the analyzer that captured
that data.
or incomplete.
Setting searchability options for
analyzers
Annotate data with custom
text.
None. You can define custom fields and
create analyzers to insert the custom
field into the database.
Managing custom fields
Generate custom metrics.
None. You can create analyzers to
generate and update custom metrics.
Configuring analyzers
Migrate hit analysis filters and
settings from Foglight
Experience Viewer (FxV)
None. You can export your FxV hit
analysis settings and import them as
traffic analysis settings.
See “Importing hit analysis
configurations from Foglight
Experience Viewer” in the online
help
Focusing on selected web applications
and web sites
You can often improve the usability of your data simply by focusing your data capture only on the web
applications and web sites that you want to monitor. For example, you may not be interested in monitoring a
web email application, but the hits associated with that application are cluttering your search results. Many
customers choose to monitor traffic to and from selected web servers and application servers only. Another way
to focus data capture is to avoid collecting data on subnetworks that carry intranet web traffic that you are not
interested in.
This section covers the following topics:
•
Monitoring selected IP addresses only
•
Excluding subnets
Monitoring selected IP addresses only
By default, a Sniffer monitors all web traffic crossing its network tap point. This includes content from all server
IP addresses carrying traffic on the monitored ports. To monitor only the subset of IP addresses that you have
identified, you need to set each Sniffer to filter by IP address.
IMPORTANT: You need to keep the list of server IP addresses up-to-date as new servers are added to your
monitored environment.
To monitor selected IP addresses only:
1
Specify the IP addresses you want to monitor. For instructions, see Step 2: Specifying monitored IP
addresses.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
25
2
Click APM > Configure > Traffic Capture > Sniffers.
3
In the row for a Sniffer, click Edit
4
Select the Filter by IP Address check box.
5
Click Save.
6
Repeat for all other registered Sniffers.
.
Excluding subnets
If you are not interested in capturing data for a subnet in your network, you can exclude the subnet by defining
a Subnet Filter. Sniffers ignore all traffic associated with the subnet, which means the collected metrics no
longer reflect hits and sessions occurring within the subnet. For more information, see Excluding subnets from
data collection.
To add a subnet filter:
1
On the navigation panel, under Dashboards, click APM > Configure Advanced Options > Subnet Filters.
2
Click Add.
3
In the Add Subnet Filters dialog box, complete the options. For help with options, see Defining subnet
filters.
4
Click Save.
Assigning user roles
Now you need to control who can see the data and change the configuration. You need Foglight Administrator
privileges to add users and assign groups and roles. If you logged into an appliance-based version of Foglight
with the default user name and password, you are automatically assigned the role of Foglight Administrator, as
well as APM Administrator and APM Operator.
Table 3. APM roles and their associated privileges
User Role
APM Administrator
Privileges
Belongs to Group
Access to all Foglight APM-related
dashboards.
APM Administrators
(also includes APM Operator, Console
User, and
General Access)
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
26
Table 3. APM roles and their associated privileges
User Role
Privileges
Belongs to Group
APM Operators
APM Operator
Access to the following dashboards:
Geographical Perspective, Sequence
Explorer, Transactions, Web Sites and
Endpoints, and Search.
Access to view sensitive hit details and
sensitive content hidden by security
rules.
No groups
APM Sensitive Data Viewer
(also includes Console User and
General Access roles)
You define users and their privileges in the same way as any other user roles in Foglight. For security purposes,
you should start by changing the password for the default user foglight. Next, add users and assign them to
groups and roles.
To change the password and assign user privileges:
1
On the navigation panel, under Dashboards, click Administration > Users & Security.
2
Click Manage Users, Groups, and Roles.
3
Click foglight and select Change Password. Specify the new password and click Confirm.
4
To create users and assign privileges, follow the instructions in the online help. Open the right-hand
action panel, select the Help tab, and click the More link.
Next steps
Congratulations, you have completed the initial configuration of your Foglight APM Appliances. You can revisit
your configuration at anytime to refine your settings or make your data more meaningful. For more information,
see Configuring traffic capture and Configuring traffic analysis.
Foglight users with the APM Operator role can review the real user data collected to quickly identify
performance issues in your web sites and applications. Start from any of the top-level APM dashboards to view
aggregate metrics and drill down to more detail. Alternatively, you can run searches from the Search
dashboards to find stored data that matches your specified parameters. You can replay hits to see what your
users experienced while using your site or application. For more information about using these dashboards, see
the Foglight APM User Guide.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
27
APM Administrators can design analyzers to collect additional metrics and hit details for interesting hits.
Whenever web traffic meets the set of conditions that you defined for the analyzer, the analyzer runs the
actions you specified, such as incrementing a metric, updating a custom field, or running a script. You can also
define analyzers to collect data about the sequences in which hits occur or to expand the session details
collected for user sessions. For more information, see Configuring analyzers.
Finally, APM Administrators can use the Appliance Health dashboard to monitor the health of the Foglight APM
Appliances and their software components (Sniffers, Archivers, and Relayers). For more information, see
Maintaining and troubleshooting.
Foglight APM 5.9.9 Administration and Configuration Guide
Getting started
28
2
Configuring traffic capture
The Getting started section walked you through setting some of the traffic capture options that are important
when you first begin monitoring your web traffic. This section describes all the tasks you can perform in each of
the Traffic Capture dashboards.
NOTE: You can fine-tune your data collection and analysis by configuring the traffic analysis options and
analyzers. These topics are covered in Configuring traffic analysis and Configuring analyzers.
Traffic capture includes the following activities:
•
Managing Sniffers
•
Managing Archivers
•
Managing capture groups
•
Managing sessionizing policies
•
Managing monitored ports
•
Managing monitored IP addresses
•
Managing monitored subnets
•
Managing identifiers for virtual addressing schemes
•
Managing private keys
•
Managing web services and SOAP operations
Managing Sniffers
Sniffers capture, decrypt, and analyze web traffic, and transmit content and metrics to one or more Archivers
within the same capture group as the Sniffer. For more information about how Sniffers work, see Understanding
how Foglight captures real user data.
An installation of Foglight APM Appliances requires at least one Sniffer component. Smaller installations may
use only a single Sniffer, but larger installations usually require many Sniffers running simultaneously. When
using an All-in-One Appliance, the Sniffer component comes preregistered as the Local Sniffer—you do not need
to register it. For installations with standalone Sniffer Appliances, you need to add and configure the Sniffers.
Later on if you define additional monitored ports or decide to use monitored IP addresses, you can choose which
Sniffers monitor the web traffic running through those devices. For more information, see Defining monitored
ports and Defining monitored IP addresses.
To define and manage Sniffers, navigate to APM > Configure > Traffic Capture > Sniffers.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
29
Figure 3. Sniffers dashboard
In the Sniffers dashboard, you can add, copy, edit, and remove Sniffers using the standard controls. You can also
test Sniffers to confirm that they are configured correctly. These topics are summarized in the following
subsections. For procedures, see the online help.
In the Sniffers dashboard, you can perform the following tasks:
•
Defining Sniffers
•
Testing Sniffer communications
Defining Sniffers
When adding, copying, or editing a Sniffer definition, you can set the following options.
Table 4. Defining Sniffers
Option
Setting the Option
Name
Specify a unique name for the Sniffer.
Description
Optional—provide information about the Sniffer. For example, describe where the
Sniffer is located or which applications and sites it is primarily intended to monitor.
Sniffer Host
Specify the Sniffer’s host name as a DNS name, an IP address, or as localhost (if
appropriate).
Foglight IP Address
Specify the IP address of your Foglight Management Server. Occasionally, you may
need to specify a different IP address to connect to Foglight, such as when a Sniffer
is deployed remotely and the network uses network address translation to hide the
true IP address of the Management Server.
Capture Group
Use the Default Capture Group or select a different capture group. The Sniffer
sends the data it collects to the Archivers within its capture group. For more
information, see Managing capture groups.
NOTE: Selecting None as the capture group means the Sniffer does not send data to
any Archivers, effectively disabling the Sniffer.
TCP Connection
Timeout
By default, Sniffers stop tracking inactive monitored TCP connections after 160
seconds. If your site uses proxies that keep TCP connections open without
transmitting any data for extended periods of time, specify the TCP timeout period
enforced by the applications this Sniffer monitors.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
30
Table 4. Defining Sniffers
Option
Setting the Option
Filter by IP Address
Clear (default) to capture web traffic for all IP addresses in the monitored
environment. Select to capture only the traffic to and from the web servers whose
IP addresses are listed on the Monitored IP Addresses dashboard. For more
information, see Managing monitored IP addresses.
NOTE: In most cases, the Sniffer consumes fewer CPU and memory resources when
filtering by IP address.
Select (default) to filter monitored traffic at the kernel level. This setting uses CPU
and memory resources most efficiently. Clear to filter traffic within the Sniffer’s
Perform Traffic Filtering process rather than in the kernel. If the kernel filter becomes too large, Foglight
APM automatically begins filtering at the Sniffer level regardless of this setting.
in Kernel
NOTE: You should filter at the kernel level unless your network uses a non-standard
variant of the IEEE802.1ad standard.
Example
Figure 4. Adding Sniffer example
Testing Sniffer communications
After you register all Sniffers in your environment, click Test to verify that the Sniffers are reachable.
The Test Sniffer dialog box shows if Foglight can successfully communicate with the Sniffer, and lists any Sniffer
diagnostic messages. If any errors appear, you may need to edit a Sniffer definition.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
31
Figure 5. Test Sniffers example
Managing Archivers
Archivers capture content and metrics collected by Sniffers, analyze that content, and maintain databases used
for searching, reporting, and replay. Data comes from Sniffers assigned to the same capture group as the
Archivers. For more information about how Archivers work, see Understanding how Foglight captures real user
data.
An installation of Foglight APM Appliances requires at least one Archiver component. Smaller installations may
use only a single Archiver, but larger installations usually require many Archivers running simultaneously. When
using an All-in-One Appliance, the Archiver component comes preregistered as the Local Archiver—you do not
need to register it. For installations with standalone Archiver Appliances, you need to add and configure the
Archivers.
To define and manage Archivers, navigate to APM > Configure > Traffic Capture > Archivers.
Figure 6. Archivers dashboard
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
32
In the Archivers dashboard, you can define and remove Archivers using the standard controls. You can also test
Archivers to confirm that they are configured correctly. You can also modify some database shard settings that
affect all Archivers. These topics are summarized in the following subsections. For procedures, see the online
help.
In the Archivers dashboard, you can perform the following tasks:
•
Defining Archivers
•
Testing Archiver communications
•
Setting database shard span and retention periods
Defining Archivers
When adding, copying, or editing an Archiver definition, you can set the following options.
Table 5. Defining Archivers
Option
Setting the Option
Name
Specify a unique name for the Archiver.
Description
Optional—provide information about the Archiver. For example, describe where the
Archiver is located.
Archiver Host
Specify the Archiver’s host name as a DNS name or an IP address.
Specify the IP address of the network device that the Archiver uses to capture data
transmitted from a Sniffer. The device differs depending on how the appliances are
deployed in your network.
•
The most common deployment involves directly cabling a Sniffer to an
Archiver using the auxiliary devices (eth1) on each appliance. When your
administrator configured the Archiver appliance, he assigned an IP address
to the auxiliary device using the console program on the Archiver appliance.
•
In other deployment scenarios, Sniffers and Archivers communicate with
each other over a dedicated private switch. In this case, the Archiver
appliance captures data though its control device (eth0).
•
In an all-in-one appliance deployment, the Archiver captures data from the
local Sniffer through the appliance’s control device (eth0).
Capture IP Address
Foglight IP Address
Capture Group
Specify the IP address of your Foglight Management Server. Occasionally, you may
need to specify a different IP address to connect to Foglight, such as when a Sniffer
is deployed remotely and the network uses network address translation to hide the
true IP address of the Management Server.
Accept the Default Capture Group or select a different capture group. The Archiver
receives data from the Sniffers within its capture group. For more information, see
Managing capture groups.
NOTE: Selecting None as the capture group means the Archiver does not receive
new data from any Sniffer. For more information, see Assigning Sniffers and
Archivers to capture groups.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
33
Example
Figure 7. Adding Archiver example
Testing Archiver communications
After you register all Archivers in your environment, click Test to verify that the Archivers are reachable. If any
errors appear, you may need to edit an Archiver definition.
Setting database shard span and retention
periods
Foglight stores real user data in database shards distributed across Archivers. The shard settings available from
the Manage Archiver dashboard apply to all Archiver databases. Shard span refers to the period of time
represented by the data within a shard. Shard retention refers to how long shards are retained. For more
information, see Archiver databases.
You can set the following options.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
34
Table 6. Setting the database shard span and retention periods
Option
Setting the Option
Maximum Shard Span
Data captured by the Archivers is stored in database “shards”. The size of each
shard is limited to a fixed maximum size, as well as a maximum time span
(whichever comes first). The Maximum Shard Span setting allows that time span to
be configured. Typically, this value is set to 24 hours. It can be changed to a smaller
number when it is desirable to purge data from the system after a small period of
time (data is removed from the system one shard at a time, based on the value of
the Maximum Shard Retention option).
Maximum Shard
Retention
By default, the shard retention is zero, which means that shards are removed only
as needed to make space for new shards. To remove shards after a certain length of
time, specify the time in hours. The value must be two greater than the value of
Maximum Shard Span.
Example
Figure 8. Common Archiver settings
Managing capture groups
A capture group contains Sniffers and Archivers. The Sniffers send data to Archivers within the same capture
group using Relayers. The Relayers balance the load among the Archivers in the capture group. For more
information, see Understanding how Foglight captures real user data.
By default, Sniffers and Archivers are assigned to the Default Capture Group. You can define additional capture
groups as required.
To define and manage capture groups, navigate to APM > Configure > Traffic Capture > Capture Groups.
Figure 9. Capture Groups dashboard
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
35
In the Capture Groups dashboard, you can define and remove groups using the standard controls. You can also
test capture groups to confirm that they are configured correctly. These topics are summarized in the following
subsections. For procedures, see the online help.
In the Capture Groups dashboard, you can perform the following tasks:
•
Planning capture groups
•
Defining capture groups
•
Assigning Sniffers and Archivers to capture groups
•
Testing capture group communications
Planning capture groups
Each Sniffer and Archiver can belong to one capture group. By default, this is the Default Capture Group. You
can create as many capture groups as you need. When planning capture groups, you may want to consider the
location of the Sniffers and Archivers based on the location of the appliances.
Grouping by network location
For example, you may want to group Sniffers and Archivers based on the location of the Sniffers in the network.
Let’s say you have a set of Sniffers and Archivers (Group A) monitoring traffic for a web site and two web
applications. In another location, you have a set of Sniffers and Archivers (Group B) monitoring a different web
application that has no connection to the other site or applications. In this case, you could create two capture
groups, one for Group A and one for Group B, and assign the relevant Sniffers and Archivers to these groups. The
groups would not share data.
Grouping by geographical location
In a geographically distributed installation, where Sniffers and Archivers are deployed in different data centers,
you should create at least one capture group per data center, so that data is never inefficiently load-balanced
across a WAN link.
Defining capture groups
When adding or editing a capture group, you can set the following options.
Table 7. Defining capture groups
Option
Setting the Option
Name
Specify a unique name for the capture group.
Description
Optional—provide information about the capture group. For example, describe what
web traffic the capture group monitors.
Maximum Queue Size
Specify the maximum size of the memory buffer used to hold captured hits before
sending them to the Archivers in the capture group. The buffer allows the Sniffer to
avoid dropping hits even when they are coming in faster than the Sniffer can send
them to the Archivers. The value must be between 1,048,576 and 104,857,600.
Maximum Relayer Wait
Time
Specify the maximum amount of time (in seconds) that Relayers wait to receive
data from multiple sources (such as Sniffers or the Cartridge for Java EE
Technologies) before correlating and sending the data to the Archivers.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
36
Table 7. Defining capture groups
Option
Setting the Option
Sniffers
Displays the list of Sniffers assigned to the capture group. To add or remove a
Sniffer in a capture group, edit the Sniffer’s definition. For more information, see
Defining Sniffers.
Archivers
Displays the list of Archivers assigned to the capture group. To add or remove an
Archiver in a capture group, edit the Archiver’s definition. For more information,
see Defining Archivers.
Example
Figure 10. Adding a capture group example
Assigning Sniffers and Archivers to capture
groups
Each Sniffer and Archiver can belong to one capture group. By default, this is the Default Capture Group. If you
create additional capture groups, you can assign or reassign Sniffers and Archivers to a new capture group by
editing the Capture Group setting for the Sniffer or Archiver. For more information, see Defining Sniffers and
Defining Archivers.
Disabling Sniffers and Archivers
You can effectively disable Sniffers and Archivers by selecting None as the capture group. The Sniffer appliance
and Archiver appliance are still available, but they do not send or receive data. When an Archiver is disabled,
active Sniffers do not deliver any new data to it. The Archiver’s database, however, is still available for query
purposes.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
37
Testing capture group communications
After you create the capture groups for your environment, you can click Test to verify that the Sniffers and
Archivers are communicating. If any errors appear, you may need to edit an Archiver definition or a Sniffer
definition.
Figure 11. Testing capture group communication
Managing sessionizing policies
Sessionizing is the process by which Foglight correlates hits in the monitored web traffic with real user sessions.
Sessionizing policies describe how user sessions are identified and managed within your monitored web
applications.
You need to modify the default sessionizing policy (or create a new sessionizing policy) to include session
identifiers for your monitored web applications. For example, many applications use cookies containing session
identifiers to track user sessions. In the sessionizing policy, you can also set options that define boundaries for
sessions, such as the maximum number of hits per session, the maximum length of a session, and timeout rules.
Typically, your sessionizing policies should align closely with the session management techniques employed by
your applications.
IMPORTANT: To collect accurate and complete session details, ensure that your sessionizing policies
mirror your application session management policies. Hits containing session identifiers other than those
identified in the sessionizing policy are still captured and stored, but they do not belong to a session and
do not have any session details associated with them.
To define and manage sessionizing policies, navigate to APM > Configure > Traffic Capture > Sessionizing
Policies.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
38
Figure 12. Sessionizing Policy dashboard
In the Sessionizing Policy dashboard, you can add, copy, edit, and remove policies using the standard controls.
These topics are summarized in the following subsections. For procedures, see the online help.
In the Sessionizing Policy dashboard, you can perform the following tasks:
•
Defining sessionizing policies
For more information, see the following topics:
•
Understanding the default sessionizing policy
•
Deciding when to add more sessionizing policies
•
Understanding how sessionizing policies are applied
•
Sessionizing policies—Identifying user sessions
•
Sessionizing policies—Associating user names with user sessions
•
Sessionizing policies—Targeting sessionizing policies to URL prefixes
•
Sessionizing policies—Defining when a session has ended
Understanding the default sessionizing policy
The default sessionizing policy contains the common session identifier variables used by major programming
languages to identify user sessions. By default, Foglight looks for session identifiers under the following variable
names:
•
aspsessionid (default for ASP applications)
•
asp.net_sessionid (default for ASP.NET® applications)
•
sessionid
•
jsessionid — cookie (default for Java™ applications)
•
jsessionid — URL parameter (default for Java™ applications)
•
phpsessid (default for PHP applications)
•
siteserver.id
In many cases, the default policy is all you need. For more information, see Deciding when to add more
sessionizing policies.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
39
Figure 13. Edit Sessionizing Policy dashboard
You can edit the default sessionizing policy to add or remove session identifiers or to change the options to suit
your environment. You can also add rules to associate user names with user sessions. The URL Prefix view is
reserved for non-default sessionizing policies. For more information, see Defining sessionizing policies.
Deciding when to add more sessionizing policies
When web applications use different session identifiers to identify user sessions, you need to add the session
identifiers to a sessionizing policy. Whether you create a new sessionizing policy or update the default
sessionizing policy depends on the type of web application you want to monitor.
If you want to monitor one of the following applications, you need to create a new policy:
•
Microsoft® SharePoint®
•
Oracle® PeopleSoft
•
Oracle® Siebel
For all other types of applications, you should modify the default policy to add the session identifiers required
by your web sites and web applications. If you discover a conflict later on, you can create a new policy to
handle just the conflict situation.
Conflicts are unlikely, but they may occur. For example, if most of a site uses sessionid to pass identifiers,
but a small section of the site uses sessionid for an entirely different purpose, you need to create a new
sessionizing policy for the part of the site that is non-standard.
When defining an additional sessionizing policy, Foglight prompts you to select the type of application you want
to monitor. It uses this information to populate the Add Sessionizing Policy dashboard with session identifiers
and other settings appropriate for the selected type of application. The following table summarizes the
sessionizing policy templates:
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
40
Table 8. When you need to add sessionizing policies
Template
Description
Other
This is a blank template with no default settings.
PeopleSoft
Contains default session identifiers and username rules appropriate for Oracle®
PeopleSoft applications.
SharePoint
Contains default session identifiers and username rules appropriate for Microsoft®
SharePoint® applications.
Siebel
Contains default session identifiers and username rules appropriate for Oracle®
Siebel applications.
You can modify settings as required to suit your environment. You can also target the sessionizing policy to a
specific domain name or extended URL prefix. For more information, see Defining sessionizing policies.
Understanding how sessionizing policies are
applied
If you have only one sessionizing policy, Foglight uses the rules defined in that policy to correlate hits into user
sessions. When you have more than one sessionizing policy, Foglight first checks the URL of the hit against the
URL prefixes defined in non-default sessionizing policies. If there is a match, the matching sessionizing policy is
applied to user sessions. If they do not match, the default sessionizing policy is used.
In either case, if the user session does not contain any of the session identifiers listed in the applicable
sessionizing policy, the hits for that session are saved but do not contain session details.
Defining sessionizing policies
When adding, copying, or editing a sessionizing policy, you can set the following options.
Table 9. Defining sessionizing policies
Option
Setting the Option
Name
Specify a meaningful name for the policy.
Description
Optional—provide information about the sessionizing group. For example, describe
which web applications this policy handles.
Type
Select the template that matches the web application. For more information, see
Deciding when to add more sessionizing policies.
URL Prefixes
Non-default Sessionizing Policies Only—Optional—specify URL prefixes. In the
monitored web traffic, when a hit occurs on a URL that matches the URL prefix, the
hit is sessionized using this policy. For more information, see Sessionizing policies—
Targeting sessionizing policies to URL prefixes.
Session Identifiers
Review the list of default session identifier variables included in the selected
template Type. If your application uses different variables, you need to add them.
For more information, see Sessionizing policies—Identifying user sessions.
Username Rules
Optional—if you want to attempt to capture user names and associate them with
sessions, see Sessionizing policies—Associating user names with user sessions
Options
Options define when a session should be considered terminated. You can edit these
options to suit your monitored web applications. For more information, see
Sessionizing policies—Defining when a session has ended.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
41
Example
Figure 14. Adding sessionizing policy example
Sessionizing policies—Identifying user sessions
Foglight checks the session identifier to determine when a user session starts and to capture and organize
session details for all hits that share the same session identifier. If your web applications use session identifier
variables that are not currently included in a sessionizing policy, you need to add them. If you do not specify
them, Foglight cannot capture session details for hits containing the unidentified session identifier variables.
Typically, the session identifiers are located in cookies. Session identifiers can also be found in URLs (in the
path, as a parameter, or as query variable), request fields (as POST parameters), and client IP addresses (used
by some intranet applications). Each session requires a distinct session identifier.
To find out which session identifier variables are used by your in-house applications, you can ask your
application developers. For session identifiers located in cookies, you can use the built-in Discovery tool to
identify likely candidates. Finally, you can use a freely available tool, such as Mozilla® Web Developer Add-on,
to help you identify the session identifier variables.
NOTE: ASP .NET® applications typically append numbers to the session identifier prefix aspsessionid.
When you include the aspsessionid variable in a sessionizing policy (default behavior), Foglight
automatically sessionizes all cookies that begin with the string aspsessionid. You do not need to add
each numeric variation on aspsessionid as a separate entry.
To specify session identifier variables, navigate to APM > Configure > Traffic Capture > Sessionizing Policies,
and expand Session Identifiers.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
42
Figure 15. Session Identifiers view
In the Session Identifiers view, you can add, edit, and remove session identifier variables using the standard
controls. You can use the Discovery tool to find cookies that contain session identifiers. You can also change the
priority order of the identifiers using the arrow icons. These topics are summarized in the following subsections.
For procedures, see the online help.
In the Session Identifiers view, you can perform the following tasks:
•
Discovering session identifier variables in cookies
•
Defining session identifiers
•
Prioritizing session identifiers
Discovering session identifier variables in cookies
When you run the Discovery tool from the Session Identifiers view, it creates a list of candidate cookies. Your
monitored applications may or may not be using these cookies for session management. You need to select the
ones used by your applications. The tool updates the list continuously until you stop the discovery process.
Figure 16. Session Identifiers Discovery dialog box
The following table summarizes the information in the Session Identifiers Discovery dialog box.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
43
Table 10. Session Identifiers Discovery information
Column Name
Description
Cookies
A cookie that may contain a session identifier. The session identifier variable
appears as the cookie name. For more information, see Discovering session
identifier variables in cookies.
The number of times the cookie was found in the traffic.
Hit Count
# Distinct Values
Variation
TIP: Cookies with a low hit count are probably not being used as session identifiers.
Look for high hit counts.
The number of unique values associated with the cookie that were found in the
traffic.
A ranking that indicates the percentage of unique values associated with the
cookie.
TIP: Session identifiers require a unique value for each unique user session, so look
for cookies with a high variation percentage.
Real values associated with the cookie. Displays up to three of the most used
values. Displays a maximum of ten characters per value.
Sample Values
TIP: If you know the pattern used for session identifier values in your application,
you can confirm that the values in the Sample Values column conform to that
pattern.
The most likely candidate cookies appear at the top of the discovery list with their check boxes selected by
default. The Variation and Sample Values columns can be particularly helpful in determining the cookies that
may contain session identifiers. When you stop the discovery process, you can order the results by the Variation
column to identify cookies with a high percentage of unique values. You can use the sample values to confirm
that you are looking at the right cookie for your application.
Identifying session identifiers in URL paths
Some applications use URL rewriting to embed a session identifier in the URL path. For example, the following
URL contains the session identifier value 123A456789B012:
•
http://www.quest.com/this-topic/123A456789B012/that-action.do
If one of the applications you want to track uses URL rewriting, you need to define a session identifier of type
URL Path and specify a regular expression to match the session identifier portion of the URL path. For more
information, see Defining session identifiers.
Example: Matching and extracting session identifiers
Consider the following URLs:
•
www.example.com/products/sid=ABC/content/page1.html
•
www.example.com/products/sid=XYZ/content/page2.html
The session identifier variable is sid, and it is embedded after products/ and before /content. In the Regular
Expression box, we use the following regular expression as the pattern to match:
.*/sid=(.*)/content.*
The parentheses show the part of the path to be used as the session identifier. In the example URLS, the session
identifiers are ABC and XYZ.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
44
Figure 17. Adding session identifier example
Defining session identifiers
When adding or editing a session identifier variable, you can set the following options.
Table 11. Defining session identifiers
Option
Setting the Option
Specify a unique name for the session identifier.
Name
NOTE: When Type=URL Path, the value in the Name field is not used by the
Archiver. However, you should specify a name for administration purposes.
Description
Optional—provide more information about the session identifier variable. For
example, you can include the name of the application that uses the variable.
Type
Select the type of session identifier variable you are defining. For example, select
Cookie if session identifiers are located in cookies.
Regular Expression
When Type=URL Path, specify the pattern for the embedded session identifiers as a
regular expression. To test your expression, click
to open the Extraction Rule
dialog box. For more information, see “Defining extraction expressions” in the
online help. For an example, see Identifying session identifiers in URL paths.
Prioritizing session identifiers
After you finish adding your session identifiers, you should arrange them in priority order from top to bottom in
the table. If a hit contains more than one configured session identifier—for example, some sites use different
tools to develop different sections of the site which may result in multiple identifiers—the order of the session
identifier variables in the table tells the system which session identifier to use to track the current session. You
can change the location of a variable within the table by clicking its Up
or Down
icon.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
45
Figure 18. Session Identifiers view
Sessionizing policies—Associating user names
with user sessions
Many web applications prompt end users to log into the application using a user name and password. When the
user enters the login credentials on the web page, the application transmits the user name to the application
server, typically in a form (or POST) parameter, but potentially in an XML variable, cookie, or HTTP header.
Foglight can extract the user name from the web traffic and associate the user name with the user session.
NOTE: Username rules do not affect how user sessions are identified; they only affect how sessions are
labeled in the Session Explorer, search results, and reports.
To include user names in your session details, navigate to APM > Configure > Traffic Capture > Sessionizing
Policies, and expand Username Rules.
Figure 19. Username Rules view
In the Username Rules view, you can add, edit, and remove username rules using the standard controls. You can
also use the Discovery tool to find cookies that contain user names. These topics are summarized in the
following subsections. For procedures, see the online help.
In the Username Rules view, you can perform the following tasks:
•
Discovering user names
•
Defining username rules
For more information, see the following topics:
•
Capturing user names in a single sign-on environment
•
Identifying user names in XML
•
Understanding missing user names
Discovering user names
When you run the Discovery tool from the Username Rules view, it creates a list of candidate variables. Your
monitored applications may or may not be using these variables for user names. You need to select the ones
used by your applications. The tool updates the list continuously until you stop the discovery process.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
46
Figure 20. Username Rules Discovery dialog box
The following table summarizes the information found in each column of the Username Rules Discovery dialog
box.
Table 12. Username Rules Discovery
Column Name
Description
A variable that may contain a user name.
Variable
TIP: If you see the Authorization variable, your network includes a single sign-on
environment for end users. For more information, see Capturing user names in a
single sign-on environment.
The number of times the variable was found in the traffic.
Instances
Sample Values
TIP: Variables with low number of instances are probably not being used as a login
variable. Look for high instance counts.
Real values associated with the variable. Displays up to three of the most used
values.
The Discovery tool has an algorithm that attempts to determine which fields may contain user names. If your
field does not show up in the results, you can configure it manually. For more information, see Defining
username rules.
Capturing user names in a single sign-on environment
Foglight can match user names to user sessions in a single sign-on environment.
NTLM
NTLM is a suite of authentication and session security protocols used in various Microsoft® network protocol
implementations. Originally used for authentication and negotiation of secure DCE/RPC, NTLM is now used
throughout Microsoft's systems as an integrated single sign-on mechanism. It is recognized as part of the
Integrated Windows Authentication stack for HTTP authentication; however, it is also used in Microsoft
implementations of SMTP, POP3, and IMAP (all part of Exchange). Sniffers decode NTLM authentication
exchanges between clients and web servers, and extract the user name that end users enter when logging into
an NTLM-enabled system.
To enable Foglight to capture the user names, all you need to do is add a username rule called Authorization.
No other action is required. If you use the Discovery tool, the Authorization username shows up in the list
whenever NTLM exchanges are found in the monitored traffic.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
47
CA SiteMinder
CA SiteMinder is a centralized web access management system that enables user authentication and single signon, policy-based authorization, identity federation, and auditing of access to web applications and portals. To
monitor applications that use SiteMinder, you need to configure Foglight so that it performs the same type of
user authentication with CA SiteMinder that your applications currently perform with SiteMinder. For more
information, see Integrating with CA SiteMinder.
To enable Foglight to capture the user names, add a username rule with the same name as the session cookie
used by SiteMinder. This is usually SMSESSION or GMWSESSION. For more information, see Defining username
rules
Identifying user names in XML
When creating a username rule for XML variables, you need to specify the variable within the context of its root
element. Use a period to separate the elements, as in root.child.
For example, if you have the following XML request body:
<body>
<variable1>not_a_username</variable1>
<variable2>userXYZ</variable2>
</body>
You can create a username rule to capture the contents of variable2 (which contains the user name) using
this format:
body.variable2
In this example, the rule extracts userXYZ as the username.
Figure 21. Adding username rule example
Understanding missing user names
In the Session Explorer, you may find instances where the user name is missing from the results even though you
created a username rule. Foglight can capture user names only if users are required to log in using a web page
at the beginning of each session. Some applications, like Siebel, cache user credentials, which means users do
not need to log in every time. In these cases, the user name is not available from the web traffic and therefore
Foglight cannot associate the user name with these sessions.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
48
Another reason user names may be missing is if the application presents the login page before assigning a cookie
that you are using as a session identifier. In this case, the user name is not associated with the user session and
therefore is unavailable in the web traffic.
Defining username rules
When adding or editing username rules, you can set the following options.
NOTE: Recall that in URLs, characters that fall outside the ASCII character set are encoded. For example,
a space is encoded as %20 and the dollar sign ($) is encoded as %24. When specifying variable names
found in a URL, use the encoded value for these characters. For a list of encodings, see http://
www.w3schools.com/TAGS/ref_urlencode.asp.
Table 13. Defining username rules
Option
Setting the Option
Specify the name of the variable used to parse user names.
Name
•
When Type=Request Field, specify the variable name to match in the POST
request field or form. Encode all non-ASCII characters (see the preceding
note).
•
When Type=XML Variable, include the context for the variable, such as
body.variablename. For more information, see Identifying user names in
XML.
•
When Type=NTLM or Type=SiteMinder Cookie, see Capturing user names in a
single sign-on environment.
Description
Optional—provide more information about the variable. For example, you can
include the name of the application that uses the variable.
Type
Select the type of variable you are defining. The most commonly-used type is
Request Field, which is the type used for matching variable names found in POST
request fields.
Regular Expression
Optional—specify a pattern to match and extract user names from a field that
contains extra information. To test your expression, click
to open the
Extraction Rule dialog box. For more information, see “Defining extraction
expressions” in the online help.
SiteMinder Servers
If your environment includes CA SiteMinder Servers, select whether you want to
apply this rule to all servers (default) or to a selected server (when your servers use
different variables for user names). For more information, see Capturing user
names in a single sign-on environment.
Sessionizing policies—Targeting sessionizing
policies to URL prefixes
You can target a non-default sessionizing policy to domain names with or without subdirectories. For example,
if you want to monitor all hits associated with your web site, you can create a URL prefix policy with the site’s
domain name, such as quest.com. You then only need to add the session identifier variables and user name
variables relevant to this domain.
IMPORTANT: You cannot add URL prefixes to the default sessionizing policy. The default policy applies to
all web traffic that does not match the non-default sessionizing policies.
To specify URL prefixes, navigate to APM > Configure > Traffic Capture > Sessionizing Policies, and expand
URL Prefixes.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
49
Figure 22. URL Prefixes view
In the URL Prefixes view, you can add, edit, and remove URL prefixes using the standard controls. You can also
use the Discovery tool to display a list of possible URL prefixes observed in the monitored environment. These
topics are summarized in the following subsections. For procedures, see the online help.
In the URL Prefixes view, you can perform the following tasks:
•
Discovering URL prefixes
•
Defining URL prefixes
Discovering URL prefixes
When you run the Discovery tool from the URL Prefixes view, it creates a list of URL prefixes accessed within the
monitored environment. The tool updates the list continuously until you stop the discovery process.
Figure 23. URL Prefixes Discovery dialog box
Defining URL prefixes
When adding or editing URL prefixes, you can set the following options.
Table 14. Defining URL prefixes
Option
Setting the Option
URL Prefix
Specify the domain name—with or without subdirectories—that you want to
monitor, omitting the server name (such as ftp or www).
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
50
Example
Figure 24. Add URL Prefix dialog box
Sessionizing policies—Defining when a session
has ended
For each sessionizing policy, Foglight uses a set of options (with default values) to determine when a user
session ends. If you find that sessions are ending before you think they should be, you should consider modifying
the default values to better suit your environment.
To modify sessionizing policy options, navigate to APM > Configure > Traffic Capture > Sessionizing Policies,
and expand Options.
Figure 25. Sessionizing Policies > Options view
When a session exceeds at least one of the values specified in the Options view, Foglight considers the session
terminated. The following table describes the options that define when a session should be considered
terminated.
Table 15. Setting the sessionizing policy options
Option
Setting the Option
Maximum Hits Per
Session
Specify the maximum number of hits (of any type) that can belong to a session.
Maximum Session
Duration
Specify (in minutes) the maximum length of time a session can run.
Categorize session a
“Long”
Specify the criteria used to determine a long session for timeout purposes. Set the
number of hits that must be received and describe the type of hits (that is, HTML
hits or Hits of any type).
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
51
Table 15. Setting the sessionizing policy options
Option
Timeout “Long”
Sessions
Setting the Option
Specify (in seconds) the length of time that a long session waits to receive hits of
the specified type. Use the same session timeout period that your applications
enforce.
TIP: Generally, you should only need to adjust this setting if your users tend to have
long periods of time in which they leave an application inactive. By increasing the
timeout period, you can cause user sessions that would have been regarded as
distinct to be combined into one record.
Timeout “Short”
Sessions
Specify (in seconds) the length of time that a short session waits to receive hits of
the specified type. A short session is any session that does not qualify as a long
session. Use the same session timeout period that your applications enforce.
Page Timeout
Specify (in seconds) the length of time to wait for activity on a page.
Managing monitored ports
Monitored ports are TCP ports used by web applications to provide content to end users. Sniffers can detect
both HTTP or HTTPS traffic on monitored ports. When you first configure Foglight APM Appliances, you need to
identify the ports used by the web applications that you want to monitor.
To define and manage monitored ports, navigate to APM > Configure > Traffic Capture > Monitored Ports.
Figure 26. Monitored Ports dashboard
In the Monitored Ports dashboard, you can add, edit, and remove ports using the standard controls. You can use
the Discovery tool to identify and select active ports. Finally, you can view monitored ports by Sniffer. These
topics are summarized in the following subsections. For procedures, see the online help.
In the Monitored Ports dashboard, you can perform the following tasks:
•
Discovering ports
•
Defining monitored ports
Discovering ports
When you run the Discovery tool from the Monitored Ports dashboard, it creates a list of active ports detected
within the monitored environment. The tool updates the list continuously until you stop the discovery process.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
52
Figure 27. Port Discovery for “Sniffer” dialog box
The following table summarizes the information found in each column of the Port Discovery dialog box. While
the discovery process is active, the cumulative values start counting from the beginning of the discovery process
and, at any given moment, reflect the current value as detected in the monitored traffic for each port. When
you stop the process, the values reflect the last values collected for the ports.
Table 16. Discovering ports
Column Name
Description
Port
A list of ports in use during the discovery process.
The number of packets transmitted from clients to server via the port.
Client Packets
TIP: If the Client Packets count is zero and the Server Packet value is greater than
zero, the network tap is not configured correctly. Contact your Network
Administrator.
The number of packets transmitted from server to clients via the port.
Server Packets
TIP: If the Server Packets count is zero and the Client Packet value is greater than
zero, the network tap is not configured correctly. Contact your Network
Administrator.
The number of new TCP connections associated with the port.
New Connections
TIP: If the number of new connections equals zero or the number is small in
relation to the number of active connections, a load balancer is maintaining a
limited number of TCP connections to the web server and using a proxy server to
send all requests over the TCP connections.
Active Connections
The number of active TCP connections associated with the port.
Protocol
The protocol used by the port, such as HTTP or HTTPS.
Defining monitored ports
When adding or editing a monitored port, you can set the following options.
Table 17. Defining monitored ports
Option
Setting the Option
Port
Specify the number of the port.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
53
Table 17. Defining monitored ports
Option
Setting the Option
Protocol
Select the type of traffic occurring on this port: HTTP or HTTPS.
Associated Sniffers
Select the Sniffers to monitor this port. Generally speaking, you should add all
Sniffers to all ports. Each Sniffer is connected to a different area of your network,
so the Sniffers see different traffic. By enabling the ports used by the monitored
applications for all Sniffers, you can be sure that each Sniffer captures all the
traffic you want to monitor.
Example
Figure 28. Add Monitored Port dialog box
Managing monitored IP addresses
Monitored IP addresses are IP endpoints that provide web application content to end users. These IP addresses
can be provided by physical or virtual web servers. By default, Sniffers monitor all IP addresses sending or
receiving web traffic across the Sniffer’s tap point on the network. You may prefer to monitor a subset of IP
addresses instead.
IMPORTANT: If you decide to monitor selected IP addresses, you need to keep the list of monitored IP
addresses up-to-date as new servers are added to your monitored environment.
You may need or want to add monitored IP addresses in the following circumstances:
•
You need to monitor secure (HTTPS) web traffic. In this case, you can add monitored IP addresses and
associate private keys and HTTPS-enabled ports with the IP addresses.
•
You want to focus your traffic capture on selected web sites and web applications. In this case, you can
add monitored IP addresses for the servers carrying the desired web traffic.
•
Sniffers are dropping packets. You can use the Appliance Health dashboard to assess your monitoring
load. If a Sniffer’s Packet Drop Percentage is above 1%, you may need to focus on a subset of monitored
IP addresses or investigate alternative deployment configurations.
To define and manage monitored IP addresses, navigate to APM > Configure > Traffic Capture > Monitored IP
Addresses.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
54
Figure 29. Monitored IP Addresses dashboard
In the Monitored IP Addresses dashboard, you can add, edit, and remove IP addresses using the standard
controls. You can use the Discovery tool to identify and select active IP addresses. The DNS Lookup tool enables
you to match IP addresses to their DNS name and use those names as the host names for the monitored IP
addresses. To monitor HTTPS traffic, you can assign private keys to the IP address. Finally, you can view
monitored IP addresses by Sniffer. These topics are summarized in the following subsections. For procedures,
see the online help.
In the Monitored IP Addresses dashboard, you can perform the following tasks:
•
Discovering IP addresses
•
Defining monitored IP addresses
•
Displaying host names instead of IP addresses
•
Monitoring secure (HTTPS) web traffic
Discovering IP addresses
When you run the Discovery tool from the Monitored IP Addresses dashboard, it creates a list of active IP
addresses detected within the monitored environment. The tool updates the list continuously until you stop the
discovery process.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
55
Figure 30. IP Discovery for “Sniffer” dialog box
The following table summarizes the information found in each column of the IP Discovery dialog box. While the
discovery process is active, the cumulative values start counting from the beginning of the discovery process
and, at any given moment, reflect the current value as detected in the monitored traffic for each IP address.
When you stop the process, the values reflect the last values collected for the IP addresses.
Table 18. Discovering IP addresses
Column Name
Description
IP Address
A list of server-side IP addresses in use during the discovery process.
Host Name
The host name (DNS name) associated with the IP address.
The number of packets transmitted from clients to the server IP address.
Client Packets
TIP: If the Client Packets count is zero and the Server Packet value is greater than
zero, the network tap is not configured correctly. Contact your Network
Administrator.
The number of packets transmitted from the server IP address to clients.
Server Packets
TIP: If the Server Packets count is zero and the Client Packet value is greater than
zero, the network tap is not configured correctly. Contact your Network
Administrator.
The number of new connections to the IP address.
New Connections
TIP: If the number of new connections equals zero or the number is small in
relation to the number of active connections, a load balancer is maintaining a
limited number of TCP connections to the web server and using a proxy server to
send all requests over the TCP connections.
Active Connections
The number of active connections for the IP address.
Ports
The port numbers of ports associated with the IP address.
Defining monitored IP addresses
When adding or editing a monitored IP address, you can set the following options.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
56
Table 19. Defining monitored IP addresses
Option
Setting the Option
IP Address
Specify the IP address of the server or other device that you want to monitor. You
can enter a range of addresses, such as 172.50.112.90-99, in which case the Host
Name option is ignored.
Specify the host name associated with the IP address. If you do not know the host
name, you can use the DNS Lookup tool to discover it.
Host Name
Ignore HTTPS
Associated Sniffers
TIP: If you are running other Foglight components, you should specify host names
for all monitored IP addresses in order to facilitate data linking among the
components.
•
To avoid capturing HTTPS traffic, select this option.
•
To monitor HTTPS traffic, clear this option. After you finish defining the
monitored IP address, you need to associate one or more private keys with
it. For more information, see Monitoring secure (HTTPS) web traffic.
Select the Sniffers to monitor this IP address. One Sniffer is usually sufficient, but
you can add more than one. For example, in some networks, traffic for the same IP
address passes through more than one tap point on the network. To capture all data
for the IP address, you need to include the Sniffers located at each network tap.
Example
Figure 31. Add Monitored IP Address dialog box
Displaying host names instead of IP addresses
You can choose to add the host names for your monitored IP addresses. Host names are then displayed wherever
IP addresses are usually displayed.
IMPORTANT: If you are running other Foglight components, such as the cartridge for Java EE Technologies,
you should provide the host names for all monitored IP addresses. Foglight uses host names to associate
data collected by Foglight APM Appliances with the data collected by other Foglight components.
You can use the DNS Lookup tool to discover the host names for your monitored IP addresses. The tool searches
your network’s DNS servers to find the host names. If the DNS Lookup tool does not locate any host names,
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
57
contact your Network Administrator. Foglight APM Appliances may need to be updated to include the current IP
addresses of the DNS servers in your network.
Figure 32. DNS Lookup dialog box
You can specify host names for all monitored IP addresses, or you can specify them for individual IP addresses by
selecting their check boxes first.
Monitoring secure (HTTPS) web traffic
To analyze HTTPS traffic, Sniffers need to be able to decrypt it. You need to list the IP addresses that handle
HTTPS traffic and then associate private keys and HTTPS-enabled ports with each IP address.
TIP: If you do not want to analyze HTTPS traffic, you should avoid capturing it. Edit the monitored IP
address and select the Ignore HTTPS option.
To monitor HTTPS traffic on an IP address, you need to complete the following tasks:
1
Register HTTPS-enabled ports used by your application. Port 443 is registered by default. If you need
to register other HTTPS-enabled ports, see Managing monitored ports.
2
Specify the IP addresses that require private keys. In the Monitored IP Addresses dashboard, add IP
addresses that use the HTTPS-enabled ports you specified. You only need to specify the IP addresses
carrying secure traffic you are interested in decrypting. For more information, see Discovering IP
addresses or Defining monitored IP addresses.
3
Provide private keys. Private keys can be stored locally or hosted in an HSM server. See Managing
private keys.
4
Associate monitored IP addresses with private keys and ports. In the Monitored IP Addresses
dashboard, click Private Keys
icon in the row containing an IP address that handle HTTPS traffic. In
the dialog box, select the private key and HTTPS-enabled port required by the selected IP address. For
detailed instructions, see “Associating private keys with IP addresses” in the online help.
Managing monitored subnets
Monitored subnets enable you to track metrics for your intranet applications by subnetwork. A subnet (or
subnetwork) is a subdivision of a network where the IP addresses of all devices share a common prefix. Using
subnet masks, networks are divided into subnets based on security, performance, and physical requirements. If
you are monitoring intranet applications, you may want performance metrics broken out by subnet.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
58
To define and manage monitored subnets, navigate to APM > Configure > Traffic Capture > Monitored Subnets.
Figure 33. Monitored Subnets dashboard
In the Monitored Subnet dashboard, you can add, edit, and remove monitored subnets using the standard
controls. You can use the Discovery tool to identify and select active subnets. These topics are summarized in
the following subsections. For procedures, see the online help.
In the Monitored Subnet dashboard, you can perform the following tasks:
•
Discovering subnets
•
Defining monitored subnets
Discovering subnets
When you run the Discovery tool from the Monitored Subnet dashboard, it examines monitored network traffic
and constructs possible subnets that may be in use on your network. The Discovery tool cannot determine the
exact subnets because that information is not available in the traffic. Instead, it uses client IP addresses
extracted from IP headers in the traffic or from the x-forwarded-for HTTP request header, and constructs a
class C subnet that might be in use on your network. You may need to edit the mask portion of the subnet to
match your actual subnets. The tool updates the list continuously until you stop the discovery process.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
59
Figure 34. Subnet Discovery dialog box
The following table summarizes the information found in each column of the Subnet Discovery dialog box. While
the discovery process is active, the cumulative values start counting from the beginning of the discovery process
and, at any given moment, reflect the current value as detected in the monitored traffic for each subnet. When
you stop the process, the values reflect the last values collected for the subnets.
Table 20. Discovering subnets
Column Name
Description
Subnets
A list of subnets in use during the discovery process.
The number of packets transmitted from clients to the server through a port within
the subnet.
Client Packets
TIP: If the Client Packets count is zero and the Server Packet value is greater than
zero, the network tap is not configured correctly. Contact your Network
Administrator.
The number of packets transmitted from the server to clients through a port within
the subnet.
Server Packets
TIP: If the Server Packets count is zero and the Client Packet value is greater than
zero, the network tap is not configured correctly. Contact your Network
Administrator.
Ports
The port numbers of ports handling packets within the subnet.
You can select all the displayed subnets, or you can specify individual subnets by selecting their check boxes
first.
Defining monitored subnets
When adding or editing a monitored subnet, you can set the following options.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
60
Table 21. Defining monitored subnets
Option
Setting the Option
Name
Specify a unique name for the subnet. When selecting subnets from a discovery list,
a default name is created using the IP address and subnet mask.
Optional—select the browse button to select a type of location (City, Region,
Country). You can then choose to monitor one city, region, or country.
Geographic Location
CIDR Block
NOTE: Configuring a subnet by location makes the data for that subnet available for
map-based dashboards in Foglight. When using the geographical location, traffic is
categorized by location based on a subnet’s address.
Specify an IP address and subnet mask, separated by a forward slash
(/). For example: 177.18.1.0/7
Example
Figure 35. Add Monitored Subnet dialog box
Managing identifiers for virtual
addressing schemes
Identifiers enable you to expose IP addresses or host names concealed by virtual addressing schemes. For
example, a proxy server may conceal the originating IP addresses of clients, while a load balancer may replace
the IP addresses of the servers using a monitored application. While virtual addressing is important for security
purposes, this practice prevents Foglight from capturing and storing data with the originating client or server IP
address, which in turn makes the data you see less meaningful. In this case, you need to configure identifiers to
extract the originating IP addresses from the tags of HTTP headers. You may also need to configure your web
servers to insert the IP addresses into the headers.
NOTE: To define session identifiers, see Managing sessionizing policies.
To define and manage identifiers, navigate to APM > Configure > Traffic Capture > Identifiers.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
61
Figure 36. Identifiers dashboard
In the Identifiers dashboard, you can add, edit, and remove identifiers using the standard controls.
In the Identifiers dashboard, you can perform the following tasks:
•
Defining identifiers
For more information, see the following topics:
•
Capturing originating client IP addresses
•
Capturing server IP addresses
•
Capturing host names
Defining identifiers
Before defining identifiers, read more about the different identifiers in the following sections:
•
Capturing originating client IP addresses
•
Capturing server IP addresses
•
Capturing host names
When adding or editing a client identifier, server identifier, or host identifier, you can set the following options.
Table 22. Defining identifiers
Option
Setting the Option
Name
Specify the HTTP header that contains the IP address or host name. You can use the
browse button to select a request header (client identifiers) or a response header
(server and host identifiers) from a list.
Description
Optional—summarize the purpose for this identifier.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
62
Capturing originating client IP addresses
In typical network environments, client traffic is routed through a proxy server that replaces the originating
client IP addresses with the IP address of the proxy server. You can configure client identifiers to capture the
originating client IP addresses. Client identifiers reference HTTP request headers that contain the IP addresses
of the client computers accessing your applications.
By default, client identifiers come pre-configured for the following commonly used headers:
•
client-ip
•
client-iv
•
true-client-ip
•
x-forwarded-for
These request headers are industry standards for identifying the originating IP address of a client connecting to
a web server through an HTTP proxy server or other virtual addressing scheme. If your environment uses
different request headers, you need to configure additional client identifiers.
To capture originating client IP addresses, you need to complete the following tasks:
•
Locate the request header used to pass the client IP address; you may be able to find the headers by
running a Search Hits query, opening the Hit Detail View, and reviewing the values in the Request
Headers pane. For more information, see “Finding IP addresses in HTTP headers” in the online help.
•
Configure a client identifier with the same request header used in your environment. For more
information, see Defining identifiers.
Capturing server IP addresses
In many networks, client requests and server responses are routed through a load balancer and/or reverse
proxy. A load balancer typically uses a virtual addressing scheme to strip out server IP addresses in the IP layer
of any communication it receives from its server farm. While this behavior is important for security purposes, it
means that Foglight attributes the data for all web servers to the load balancer IP address. Usually, you want
data attributed to individual web servers.
To capture server IP addresses, you need to complete the following tasks:
•
Configure each web server in the server farm to insert an extra shared HTTP response header into the
traffic, such as SERVER-ID, and assign a unique, fictitious (for security purposes) IP address to this
header for each web server. For information about how to insert a response header tag into outgoing
traffic, see your web server documentation.
•
Configure a server identifier with the same response header. For instructions, see Defining identifiers.
For example, you can configure a web server to use the response header SERVER-ID with an IP address of
177.18.1.89. In Foglight, you configure a server identifier called SERVER-ID. Now when you view data, you
see metrics for IP address 177.18.1.89 instead of the load balancer’s IP address. The new server IP addresses
are used wherever IP addresses appear in the interface, such as when running the Discovery tool from the
Monitored IP Addresses dashboard. For more information, see Discovering IP addresses.
Capturing host names
Host identifiers are HTTP response headers that contain the host names of web servers that provide content for
your monitored applications. Archivers insert the host name into all URLs. In some monitored environments, you
may need to configure host identifiers for the session replay feature to work properly.
To create host identifiers, you need to complete the following tasks:
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
63
•
Configure each web server to insert a response header that contains the server’s host name (for
example, quest.com). For information about how to insert a response header tag into outgoing traffic,
see your web server documentation.
•
Configure a host identifier with the same response header. For instructions, see Defining identifiers.
Managing private keys
Private keys enable the system to monitor encrypted web traffic. To decrypt HTTPS traffic, Sniffers need access
to the same private keys used to encrypt the data. Private keys can be stored locally or in a Hardware Security
Module (HSM) appliance.
To load and manage private keys, navigate to APM > Configure > Traffic Capture > Private Keys.
Figure 37. Private Keys dashboard
In the Private Keys dashboard, you can add, edit, and remove local and HSM private keys using the standard
controls. These topics are summarized in the following subsections. For procedures, see the online help.
In the Private Keys dashboard, you can perform the following tasks:
•
Managing local private keys
•
Managing HSM private keys
Managing local private keys
For each secure IP address that you want to monitor, you need to export the keys as encrypted private key files.
When adding a private key in Foglight, you need both the encrypted key file plus the passphrase used when
encrypting the key file.
Supported encryption technologies
Before you begin adding private keys, you need to ensure that Foglight APM supports the same encryption
technologies in use within your network environment. Private keys need to be encrypted using a supported
encoding format. In addition, the Sniffers and your secure servers need to use the same protocols and
algorithms for secure communication. They need to establish a session using the same cryptographic protocol
(such as TLS) and the same key exchange algorithm (such as RSA). They also need to use the same bulk
encryption algorithm (such as RC4) for transmitting secure data.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
64
The following table summarizes the supported technologies.
Table 23. Supported encryption technologies
Encryption
Supported Technologies
Private Key Encoding
Cryptographic Protocols
Key Exchange Algorithm
1
Bulk Encryption Algorithm
•
PEM (Privacy Enhanced Mail)
•
DER (Distinguished Encoding Rules)
•
PKCS12
•
SSLv2
•
SSLv3.0
•
TLSv1.0
•
TLSv1.1
•
TLSv1.2
RSA
•
Camellia
•
RC4
•
RC2
•
IDEA
•
DES
•
3DES
•
AES
•
AES-GCM
•
FRT (Fortezza)
1
Diffie-Hellman is not supported because this key exchange algorithm cannot be decrypted by thirdparty monitoring tools. If you want to monitor the SSL traffic protected by Diffie-Hellman, disable all
versions of Diffie-Hellman on the web server (or SSL endpoint) using the following syntax:
!DH:!ADH:!DHE
Understanding how local private keys are stored
When you add private keys, Foglight APM Appliances encrypt the key files and save them in hidden files on the
appliance. The appliance then implements extensive hardening and security measures, ensuring that your
private keys cannot be compromised. Passphrases are not saved.
Exporting local private keys
When exporting the private key from your web server, make sure to select the Yes, Export the private key
option. Depending on the web server, the private key is not always included in an export by default. You will
need to specify a passphrase for use with the encrypted key file. For more information, see “Exporting keys
using Microsoft Management Console” in the online help.
Example
-----BEGIN RSA PRIVATE KEY----Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,F85757828BFF54C8
QlHfWyCnqpe5ag4LgNiRQaqrcTC5bBV3yT35tRbB0WKB3VsaDcHuhjlyz3ohQmgpHoZtWcCxCRm8DOROlBP
EBhmRgUSYxByyt/
Y8OvL9Ei2YFdRjBapsJjEjpTEl6AXNGHKjHbmyCHs5O1LvnwEv13b51Q0RHRpRZX1yNVL34cz/
efMCfVlhgqlYCHFb0qRzLcNFVW6vRXWfKkM5jbw67WmKjqeDa+VMMXskPRDxFVYud/
HoB+gnMP0ecnSX7k4xZrLkIwk4QcVaJST2BPiDq0DhBdUfXRurJd9AQuAECAw6SgumqIRY/
CrH31w5dxqXzs2UY0lqODpU3tVqZW8+OxX34ojsBPeH0zmeOxnmQ8IebRyex8MTHhEVnpIU4DDNdKlbE0/
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
65
seoyNGJRXFh2lDcrXlkTMgrOVg1VL216vGIgbpfyZgOBDiexkEj24tW4dO0iRhyqiOZVZVlR2kcm9L64Nru
hwgzLAGPDlaGfLkExATQvp1NNETxoR0iMe4vBgA1xLZDZ/atH4U/
oE63Hedj4B9C3MG8Wapnx5lJaA7DVKSkPwKZRP5Qxcbun3R084j+Q7KwmkiOk0yT8RlQapLm7SaYahOklJP
y0eHpfo4sZPELhkU39V4smBa1LlAKCbxTzFYq22CLCKBtEkCKrc9yfOQ8KpQ9SsD2/
VCJRfLGhMRrVzSheOXeGDA/oKq/
ZZLO27r68odZaXQw1e9dR7oKStqNOG8M3WAsY3LEH++DAubqXDJ6g00aHC2eiVr9ZFfRUpKWRqybHns+ukA
cg436UJoK2RMyhWyBHV8P2cDqF1ow9QQzO8Qg==
-----END RSA PRIVATE KEY-----
Defining local private keys
When adding or editing a local private key, you can set the following options.
Table 24. Defining local private keys
Option
Setting the Option
Key Name
Specify a unique name for the private key.
Choose File
Navigate to and select the encrypted private key file.
Passphrase
Type the passphrase used to encrypt the private key during the export process.
Example
Figure 38. Add Private Key dialog box
Troubleshooting local private keys
When adding a private key, you may see the following message:
The key file you supplied is invalid. Possible reasons are: 1) incorrect password,
2) the file does not contain a private key, 3) the file was exported with strong
encryption, or 4) the file is in an unrecognized format.
Due to the nature of SSL, it is impossible to determine precisely why your key may be invalid, but you can try
exporting the key again, making sure to select the following options:
•
Select a supported encoding format: PKCS, PEM, or DER.
•
Select the Export Private Key (or similar) option. Depending on the web server, the private key is not
always included in an export by default.
Managing HSM private keys
An HSM is an appliance that stores and manages private keys. When integrated with an HSM appliance, Foglight
can use the keys within the HSM server to decrypt HTTPS traffic, without having to store the keys locally. Before
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
66
you can define HSM private keys, you need to complete the steps in Integrating with SafeNet Hardware Security
Modules.
Supported HSMs
Foglight currently supports SafeNet Luna SA HSM appliances.
Defining HSM private keys
When adding or editing an HSM private key, you can set the following options.
Table 25. Defining HSM private keys
Option
Setting the Option
Partition
Select the HSM partition that contains the key you want to add.
Enter Password
Click Enter Password and type the password for the partition.
Key Name
Select the key.
Example
Figure 39. Add Hsm Key dialog box
Managing web services and SOAP
operations
Web services support automated machine-to-machine interaction designed to accomplish specific tasks. They
typically use XML over HTTP following the SOAP specification. Web services provide one or more SOAP
operations (or methods) that provide specific functions to the consumer of the SOAP service. When a Sniffer
processes a hit containing a SOAP message in the monitored traffic, it parses the message and extracts the
name of the SOAP operation. The extracted SOAP operation becomes a hit detail.
The Sniffer also attempts to extract the name of the web service from the SOAP message’s SOAPAction HTTP
header tag. As of SOAP v.1.2, however, the SOAPAction tag is no longer mandatory. When this tag is missing
from a SOAP message, the resulting hit includes the SOAP operation, but does not include the web service to
which the operation belongs. You can remedy this situation by using the Web Services dashboard to specify how
SOAP operations map to web services.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
67
NOTE: While unlikely, it is possible for more than one web service to have SOAP operations with the same
name. In this case, the SOAP operation maps to the first web service in the list.
To manage web services, navigate to APM > Configure > Traffic Capture > Web Services.
Figure 40. Web Services dashboard
In the Web Services dashboard, you can add, edit, and remove web services from the list. You can also create
web service mappings automatically by importing WSDL files. These topics are summarized in the following
subsections. For procedures, see the online help.
In the Web Services dashboard, you can perform the following tasks:
•
Creating web services automatically from WSDL files
•
Defining web services and SOAP operations
Creating web services automatically from WSDL
files
You can create a web service mapping automatically by importing the web service’s WSDL file. When you import
the file, the Add Web Service dialog box is automatically populated with the web service name and SOAP
operations. You can modify the list of operations, if necessary. If the elements in the WSDL file use some other
prefix than the standard wsdl:message, you need to specify it.
Only the web service and SOAP operation names are saved. Foglight does not save any other information from
the WSDL file.
Defining web services and SOAP operations
When adding or editing web services, you can set the following options.
Table 26. Defining web services and SOAP operations
Option
Setting the Option
Name
Specify the name of the web service.
Description
Optional—provide more information about the web service.
Operations
Click Add. Add a list of SOAP operations performed by the web service.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
68
Example
Figure 41. Add Web Service dialog box
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic capture
69
3
Configuring traffic analysis
In the Traffic Analysis dashboard, you set the options that affect how data is updated, annotated, indexed,
secured, and displayed. This section describes all the traffic analysis options (except analyzers) and tells you
when and how to set them. Analyzers are discussed in the next major section, Configuring analyzers.
TIP: If you are not satisfied with changes that you make to the traffic analysis configuration, you can roll
back your changes through the Change Configuration Log dashboard. For more information, see Managing
configuration changes.
Traffic analysis includes the following activities:
•
Managing custom fields
•
Managing custom metrics
•
Managing security policies for sensitive hit details
•
Managing security policies for sensitive content
•
Managing pivots
•
Configuring advanced options
•
Managing domain rules
•
Storing large hit details
•
Managing special events
•
Troubleshooting improperly classified URLs
•
Managing user agent rules
•
Defining additional parsed response types
•
Defining captured request body types
•
Modifying traffic analysis options
•
Managing scripts
Managing custom fields
Custom fields are user-defined variables that are scoped for hits, sequences, or sessions. Using custom fields,
you can attached additional information to individual hits, sequences, or sessions to make them easier to find
when searching or to provide more context for the data when viewing results.
NOTE: To create a sequence-scoped custom field, you need to define it from a sequence analyzer. For
more information, see Introducing sequence analyzers.
To define hit- and session-scoped custom fields, navigate to APM > Configure > Traffic Analysis, and expand
Custom Fields.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
70
Figure 42. Custom Fields view
The following table describes the data columns contained in the view.
Table 27. Managing custom fields
Column Name
Description
Name
The name of the custom field.
References
Analyzers that update the custom field.
Scope
Scope of the custom field: Hit, Sequence, or Session.
Type
Data type of the custom field: Text or Numeric.
In the Custom Fields view, you can add, copy, edit, and remove custom fields using the standard controls.
In the Custom Fields view, you can perform the following tasks:
•
Predefined custom fields
•
Setting and updating the value of custom fields
•
Defining custom fields
Predefined custom fields
Foglight uses the following predefined custom fields to track web page views that provide boundary information
about user sessions.
•
Entry Page—identifies the first HTML page visited by the user during a session.
•
Initial Referer—identifies the web page the user was visiting prior to starting a session.
•
Last Page Visited—identifies the last HTML page visited by the user during a session.
You may find it useful to review to the predefined custom fields in the software to get a sense of how they are
defined and used.
Setting and updating the value of custom fields
To set and update the value custom fields, you need to define analyzers. For more information, see Updating
custom fields.
Defining custom fields
When adding or editing a custom field, you can set the following options.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
71
Table 28. Defining custom fields
Option
Setting the Option
Specify a unique name for the custom field.
Name
NOTE: Custom fields with different scopes can have the same name.
Description
Optional—summarize the purpose for this custom field.
Scope
Set the scope for the custom field by selecting either Hit or Session. The custom
field can be updated only by an analyzer with access to the selected scope. For
example, a hit analyzer can update both kinds of custom fields, but a session
analyzer can update only session custom fields.
Storage
Select an option that determines whether the value for the custom field is stored,
and if so, how the custom field is treated in search results. When Scope=Session,
you can select options that enable audit logging on the custom field. For a
description of the options, see Storage option.
Storage Format
Set the storage format by selecting either Numeric or Text as the data type. When
Numeric is selected, you can search using operators. For example, you can define a
search that returns the value of the custom field “shopping cart value” when the
value is greater than 100. You can also sort numeric data.
Value Assignment Mode1
Select an option that reflects how you want to handle values for the custom field.
For a description of the options, see Value assignment mode option.
Separator
Optional—when the Value Assignment Mode is set to one of the Append options, you
can define a separator to be inserted between values. Typical separators are a
comma or a colon, but you can use any character or combination of characters that
suits your needs.
Update if blank1
Select to allow the custom field to be set to an empty (or blank) value. Clear
(default) to ignore blank values, that is, an existing value is not replaced by a new
value if the new value is blank.
Truncate large values
Select (default) to limit values to 2 KB (2,048 characters). Clear to store large
values up to 64 KB.
1
1
These options can be overridden by analyzers that update the values of custom fields. For more
information, see Defining custom field updates.
Storage option
You can select any of the following values for the Storage option. For session-scoped custom fields, you can
select storage options that enable audit logging. With audit logging enabled, you can track the custom field
when Foglight users view sessions.
Table 29. Storage option values
Storage Option
Description
Available only during
traffic analysis
The value of the custom field is required while monitoring traffic, but does not
need to be stored in the Archiver database. For example, an analyzer may use the
value when matching conditions.
Show in search results
The custom field appears in search results.
Not in search results
The custom field does not appear in search results, but can be seen when
examining details for a specific hit or session (depending on the scope).
First column in search
results
The custom field appears in search results. When a hit or session has multiple
custom fields associated with it, a custom field with this storage option is listed
before other custom fields.
Show in search results
(+ audit log)
Session scope only. The custom field appears in search results. The field is flagged
for audit logging.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
72
Table 29. Storage option values
Storage Option
Description
Not in search results
(+ audit log)
Session scope only. The custom field does not appear in search results, but can be
seen when examining details for a specific hit or session (depending on Scope). The
field is flagged for audit logging.
First column in search
results
(+ audit log)
Session scope only. The custom field appears as the first column in a table of
search results. The field is flagged for audit logging.
Value assignment mode option
You can select any of the following values for the Value Assignment Mode option. The Result column shows the
final custom field value in the case where four assignments to the same field occur in the following sequence:
1, 2, 3, 2.
TIP: A custom field with its Value Assignment Mode set to “Increment value” behaves the same as a
custom metric. For more information, see Creating custom metrics.
Table 30. Value assignment mode option
Value Assignment Mode
Description
Result
Append to value on each match
Concatenates values as they are encountered. If a Separator is
defined, it is inserted between the values.
1232
Append to value on each match
(if unique)
Similar to Append to value on each match, but only unique
values are added.
123
Append to value on each match
(if unique ignoring case)
Similar to Append to value on each match (if unique), but
values like “foo” and “Foo” are considered identical.
123
Increment value
Increments a numeric value by the current value.
8
Reset value on each match
Sets the custom field to the current value.
2
Set value on first match if not
already set
Sets the custom field to the first value and never updates it.
1
Example
Figure 43. Add Custom Field dialog box
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
73
Managing custom metrics
Custom metrics are user-defined, numeric variables. If an existing metric does not satisfy your requirements,
you can create a custom metric. For example, a typical reason to create a custom metric is to count some kind
of error.
NOTE: The Metrics view lists existing custom metrics. To create a metric, you need to define an analyzer.
For more information, see Creating custom metrics.
To view a summary of all custom metrics, navigate to APM > Configure > Traffic Analysis, and expand Metrics.
Figure 44. Custom Fields & Metrics view
The following table describes the data columns contained in the view.
Table 31. Managing custom metrics
Column Name
Description
Name
The name of the custom metric.
References
Analyzers currently configured to update the custom metric.
Specifies how the value of this custom metric can be updated:
Scope
•
Analyzer—only updated by the analyzer that created the metric.
•
Sequence Event—only updated by the sequence event that created the
metric.
In the Metrics view, you can edit and remove metrics using the standard controls. For details, see Editing
custom metrics.
Editing custom metrics
When editing a metric from the Metrics view, you can set the following options.
Table 32. Editing custom metrics
Option
Name
Description
Setting the Option
Specify a unique name for the custom metric.
NOTE: Metrics with different scopes can have the same name.
Optional—summarize the purpose for this custom metric.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
74
Example
Figure 45. Edit Metric dialog box
Managing security policies for sensitive
hit details
Sensitive hit details refer to private information, such as login names and passwords, that are contained within
request fields, request headers, response headers, and cookies. For security reasons, you should avoid exposing
this content to Foglight users. You can define sensitive hit detail policies that identify private information in the
web traffic and hide that data from view.
TIP: To manage sensitive content in the body of HTML pages, see Managing security policies for sensitive
content.
To define security policies for sensitive hit details, navigate to APM > Configure > Traffic Analysis, and expand
Sensitive Hit Details.
Figure 46. Sensitive Hit Details view
In the Sensitive Hit Details view, you can add, copy, edit, and remove policies using the standard controls.
For more details, see the following topics:
•
Hiding or replacing sensitive hit details
•
Omitting sensitive hit details from the database
•
Defining sensitive hit details
Hiding or replacing sensitive hit details
You should start by identifying the types of private information you expect to find in your web traffic. Then you
can create a sensitive hit detail policy that matches based on the type and name of hit detail. For example, you
could match “password” in request headers. You can choose to hide the content (and display a blank field to
Foglight users) or specify replacement text to display instead. The following example shows how to use a
sensitive hit detail policy to match and hide passwords.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
75
Example: Matching and hiding passwords
An internal financial web application requires employees to log in using their employee number and a password.
The organization wants to ensure that the logins and passwords do not appears in searches and reports.
Therefore, they create two sensitive hit detail policies, one for each type of content. For more information, see
Defining sensitive hit details.
This example describes how they match and hide passwords. The password is contained in a request header and
identified with the name “password.” In the Add Sensitive Hit Detail dialog box, they select Request Field from
the Hit Detail Type list and the equals sign from the Operator list. Then they click the browse button beside the
Hit Detail Name box and select “password” from the list.
Figure 47. Example: Matching and hiding passwords
The Replacement Text box is left blank, which means there is no visible text in searches and reports. By
selecting the Always Sensitive check box, the organization ensures that no one in the company is able to view
the passwords.
Omitting sensitive hit details from the database
Sensitive hit details are stored in the database by default. If you do not want to store sensitive hit details, you
can define a hit analyzer with a Hit Storage Restriction policy of Do not store hit details marked “Always
Sensitive.” For more information, see Defining hit storage restrictions for hit analyzers.
Defining sensitive hit details
When adding or editing sensitive content expressions, you can set the following options.
Table 33. Defining sensitive hit details
Option
Setting the Option
Name
Specify a unique name for the security policy.
Description
Optional—summarize the purpose for this security policy.
Hit Detail Type
Select the type of hit detail, for example, Request Field.
Operator
Select an operator.
Hit Detail Name
Specify the value to match within the selected type of hit detail. You can use the
browse button to select the name from a list.
Replacement Text
Specify the text to display instead of the sensitive content.
Always Sensitive
Select to hide sensitive data from all users. Clear (default) to permit users with the
Sensitive Data Viewer role to view this sensitive data.
TIP: If you do not want to store sensitive hit details, see Omitting sensitive hit
details from the database
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
76
Example
Figure 48. Add Sensitive Hit Detail dialog box
Managing security policies for sensitive
content
Sensitive content refers to private information located in the body of HTML pages, such as credit card numbers,
social security numbers (or other government identification numbers), and passwords. For security reasons, you
should avoid exposing this content to Foglight users. You can define sensitive content expressions that identify
private information in the web traffic and hide that data from view.
TIP: To manage sensitive hit details contained within request fields, request headers, response headers,
and cookies, see Managing security policies for sensitive hit details.
To define sensitive content expressions, navigate to APM > Configure > Traffic Analysis, and expand Sensitive
Content Expressions.
Figure 49. Sensitive Content Expressions view
In the Sensitive Content Expressions view, you can add, copy, edit, and remove policies using the standard
controls.
For more details, see the following topics:
•
Hiding or replacing sensitive content
•
Omitting sensitive hit details from the database
•
Defining sensitive content expressions
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
77
Hiding or replacing sensitive content
You should start by identifying the types of private information you expect to find in your web traffic and
identifying the patterns in the content. Then you can create a sensitive content expression that uses a regular
expression to match the pattern. You can choose to hide the content (and display a blank field to Foglight users)
or specify replacement text to display instead. The following example shows how to use a sensitive content
expression to match and replace credit card numbers in the body of an HTML page.
Example: Matching and replacing credit card numbers
A grocery store web site accepts payment by credit card. After entering a card number in a request field, the
site displays a confirmation page to the end user that contains the credit card number in the body of an HTML
page. The company that owns the site does not want to expose the credit card numbers in their internal
reports.
There are actually two policies required. For the request field, they create a sensitive hit detail policy. For
more information, see Managing security policies for sensitive hit details.
For the confirmation page, they need to create a sensitive content expression. The credit card number pattern
is in the format 1234-5678-9012-3456. Therefore, the regular expression is:
([0-9]{4})\-([0-9]{4})\-([0-9]{4})\-([0-9]{4})
From the Add Sensitive Content Expression dialog box, an APM Administrator (with the Sensitive Data Viewer
role) launches the Extraction Rule dialog box to create and test the expression. As you can see in the following
image, the four sets of parentheses in the expression correspond (from left to right) to group numbers 1, 2, 3,
and 4. A special group 0 refers to the entire expression.
Figure 50. Example: Matching and replacing credit card numbers
The site owners want to replace the first three groups of numbers with XXXX and display the last group of digits.
For example, the sample credit card number would be displayed as XXXX-XXXX-XXXX-3456. The following image
shows the settings required in the Add Sensitive Content Expressions dialog box.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
78
Figure 51. Add Sensitive Content Expressions dialog box
The APM Administrator specified groups 1, 2, and 3 in the Sensitive Groups box and entered XXXX in the
Replacement Text box. The policy applies to both Requests and Responses. By selecting the Always Sensitive
check box, the owners ensure that no one in the company is able to view the credit card numbers, regardless of
user permissions.
Omitting sensitive content from the database
Sensitive data is stored in the database by default, even if it is not visible to anyone using Foglight. If you do not
want to store sensitive data, you can define a hit analyzer with a Hit Storage Restriction policy of Do not store
request or response content marked “Always Sensitive.” For more information, see Defining hit storage
restrictions for hit analyzers.
Defining sensitive content expressions
When adding or editing sensitive content expressions, you can set the following options.
Table 34. Defining sensitive content expressions
Option
Setting the Option
Name
Specify a unique name for the security policy.
Description
Optional—summarize the purpose for this security policy.
Regular Expression
Click the browse button. In the Extraction Rule dialog box, specify the regular
expression to be used to identify sensitive data. The expression may contain one or
more sets of parentheses. Each set of parentheses is implicitly assigned a group
number, from left to right, starting at 1. The entire expression is assigned group
number 0. For an example expression, see Example: Matching and replacing credit card
numbers.
Sensitive Groups
Specify group numbers from the regular expression separated by spaces. For example:
123
Replacement Text
To hide the text without replacing it, leave this box blank. To replace the text, specify
the replacement text. Each group uses the same replacement text. For example: XXXX
Content
Specify whether this policy applies to Request content, Response content, or both
Request and Response content.
Always Sensitive
Select to hide sensitive data from all users. Clear (default) to permit users with the
Sensitive Data Viewer role to view this sensitive data.
TIP: If you do not want to store sensitive data, see Omitting sensitive content from the
database.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
79
Example
Figure 52. Add Sensitive Content Expression dialog box
Managing pivots
Pivots are global categories. When you enable pivots, Foglight can store additional sets of data. You can enable
global pivot categories for hits, endpoints, and/or web sites.
To apply pivot categories to data, you need to configure analyzers that generate metrics and specify which
pivots to apply to those metrics.
For more information, see the following topics:
•
Understanding pivot categories
•
Enabling pivot categories and defining their membership
•
Enabling global pivots for web sites and endpoints
•
Working with metrics
Understanding pivot categories
The following tables summarize the pivot categories and when they may be useful.
Pivots
The general categories enable metrics basis on technology used or content type.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
80
Table 35. Pivot categories
Pivot
Browser
Browser
Category
Operating
System
Endpoint
Enables metrics by
Suitable For
The browser version used to access applications.
All
Example: Firefox® 4.0, Internet Explorer® 9.0, etc.
Environments
The browser used to access applications. Version number is ignored for this All
Environments
category.
Example: Chrome™, Internet Explorer®, Safari®, etc.
The operating system from which users accessed applications.
®
®
®
®
Example: Linux , UNIX , Windows 7, Windows XP, etc.
All
Environments
The server endpoints that are hosting the monitored web site or
All
applications. Endpoint category members are dynamically generated based Environments
on the monitored traffic.
Example:192.168.1.1, 192.168.1.2, etc.
The ContentCategory field associated with each hit. Content categories All
Environments
are based on file extension, content-type, or other criteria:
Content
Category
•
Favicon—file ends with favicon.ico
•
Download—extensions: BAT, CSV, DOC, EML, PDF, PPT, PWD, RTF, TXT,
WRI, XLS, ZIP, PKG, MSI, GZ, Z, BZ2, ISO, and JAR
•
Image—extensions: BMP, GIF, ICO, JPE, JPG, JPEG, PNG, TGA, TIF, and TIFF
•
JavaScript—extension JS
•
Media—extensions: AIF, ASF, AVI, FLV, M3U, MP3, MP4, MPA, MPEG, MP1,
MPG, MPV, PLS, RMI, RA, RM, RMJ, RP, WAV, WMA, WMV, or XPL
•
StyleSheet—CSS
•
HTML—hasHtmlTag=true
•
Instrumentation—isInstrumentationHit=true
•
SOAP—has a SOAP operation name
•
HTMLFragment—response content-type contains html
•
JSON—response content-type contains json
•
XML—response content-type contains xml
•
Other—none of the above
Location Pivots
The location pivots enable metrics by location.
Table 36. Location pivots
Pivot
Enables metrics by
Suitable For
City
The city from which users accessed monitored applications.
Internet
Country
The country from which users accessed monitored applications.
Internet
Region
The region from which users accessed monitored applications.
Internet
ISP
The Internet Service Provider (ISP) through which users accessed
monitored applications.
Internet
Subnet
The subnetwork through which users accessed monitored applications. The Intranet
category members match the list monitored subnets you defined. For more
information, see Managing monitored subnets.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
81
Enabling pivot categories and defining their
membership
You can control which pivots are globally enabled by defining the membership for each Pivot Object category on
the Traffic Analysis Pivots dashboard. For example, if your organization is not interested in tracking
performance on particular browsers, you can remove these browsers from the Browser Categories list. When a
category has no members, the pivot is disabled.
Pivot Objects settings are inherited by analyzers. You can also define which pivot categories are enabled for
endpoints and for web sites.For more information, see Enabling global pivots for web sites and endpoints.
IMPORTANT: To avoid a proliferation of unnecessary metrics, you should select only the categories and
members that provide meaningful data on the performance of your web applications and sites. For
location-based metrics, you may prefer to use the mapping feature available from search results, which
provides the same data on demand, without saving the extra pivot data.
To define and manage pivot categories, navigate to APM > Configure > Traffic Analysis, and select Pivots.
Figure 53. Pivots dashboard
From the Pivots Objects dashboard, you can expand any of the pivot categories to view its list of members.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
82
Figure 54. Browser Categories view
When you click Add, you can add and remove members by selecting them from a list.
Figure 55. Add Cities dialog box
The lists are static with the exception of the Endpoints and Subnets pivots. The Endpoints pivot is the only pivot
where the members are generate based on the IP addresses observed in monitored traffic. For the Subnets
pivot, the list is populated with the monitored subnets you specified (if any). For more information, see
Managing monitored subnets.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
83
Enabling global pivots for web sites and
endpoints
You can select the pivot categories you want enabled for web sites and endpoints. After you make your
selections, click Save.
IMPORTANT: To avoid a proliferation of unnecessary metrics, you should enable only the categories and
members that is important to understanding the performance of your endpoints and web sites.
Figure 56. Enabling global pivots for web sites and endpoints
Configuring advanced options
Advanced options are a collection of features that, generally speaking, you use only when a need arises. The
following table summarizes some situations that may require you to use these advanced options.
Table 37. Configuring advanced options
Situation/Issue
Instructions
Data for a web site is split among a number of aliases or the URL Managing domain rules
displayed is too long.
Hit detail values are truncated.
Storing large hit details
Managing scripts
You cannot perform a desired task using the traffic analysis
details and metrics provided by default. In this case, you can
extend the functionality by creating scripts, either to evaluate
analyzer conditions or to use the output of a script as a value for
custom fields and custom metrics.
You want to define global events that stop capturing data for in- Managing special events
progress sessions or sequences based on criteria defined by you.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
84
Table 37. Configuring advanced options
Situation/Issue
Instructions
Page metrics are not available for some set of URLs.
Troubleshooting improperly classified URLs
The default categorizations for the browser, browser category, Managing user agent rules
and operating system hit details do not suit your needs. For
example, if you see sessions with no values for these hit details,
it is likely that a newly-released browser needs to be
categorized.
Some sensitive content is being missed by sensitive data
Defining additional parsed response types
policies, or some content is not being keyword indexed, or some
details are not available to hit analyzers and sequence
analyzers.
You want to save request body content as request fields for
traffic analysis purposes.
Defining captured request body types
You need to change the maximum size for responses, requests,
or hit details. For example, you defined a large hit detail that
exceeds the default maximum hit detail size.
Modifying traffic analysis options
Managing domain rules
Domain rules enable you to replace a domain name with one of your choice and to aggregate data among
related domain names. The replacement domain name can be either an active domain name (one found in
monitored traffic) or a fictitious domain name. When you assign more than one domain name to the same
replacement domain name, incoming data for all original domain names are merged into a single data set under
the replacement domain name.
NOTE: When merging multiple domain names, only the data captured after the domain rule is created is
merged. Historical data is not affected.
For example, some web sites use aliases to redirect requests to another web site. If you are monitoring the web
site acorp.com, you may also be using aliases that include: acmecorp.com, acme.com, and acc.com. By
default, traffic analysis data is captured and stored separately for each alias. To report metrics accurately for
acorp.com, you should create domain rules to map all the aliases to the primary web site and thereby
aggregate the data.
You can also use domain rules to shorten the length of a URL, thereby improving the display and comprehension
of URLs in your search results. For example, if you see acme.sectors.application.corp.com in your
search results, you may want to add a domain rule to transform this site URL to acme.com.
To create domain rules, navigate to APM > Configure > Traffic Analysis, select Advanced Options, and expand
Domain Rules.
Figure 57. Domain Rules view
In the Domain Rules view, you can add, copy, edit, and remove rules using the standard controls. You can also
use the Discovery tool to identify aliases in the monitored traffic.
For more information, see the following topics:
•
Discovering aliases
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
85
•
Defining domain rules
Discovering aliases
When you run the Discovery tool from the Domain Rules view, it creates a list of potential domain rules based on
aliases. The discovery process starts by identifying the most common domain. This is considered the primary
domain name. Next, the process finds domain names that are common in structure, compares them to the
primary domain name, and constructs domain rules to map each candidate alias to the primary domain name.
As the domain rules are constructed, they are added to the list. The tool updates the list continuously until you
stop the discovery process. You can then select the rules you want to implement.
NOTE: The Discovery tool can be slow to show results for low traffic sites. Multiple domains need to be
used to access the same URL paths before the Sniffer algorithm can determine which domain names may
be aliases. You may need to wait a few minutes for results.
Figure 58. Domain Rule Discovery dialog box
The following table summarizes the information in each column.
Table 38. Discovering aliases
Column Name
Description
Find
The original domain name found in the monitored traffic.
Replace
The domain name that replaces the original domain name.
Hit Count
The number of hits associated with the original domain name.
Defining domain rules
When adding, copying, or editing domain rules, you can set the following options.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
86
Table 39. Defining domain rules
Option
Setting the Option
Find
Type the domain name, IP address, or a regular expression to match a pattern. To
select a domain name from a list of URLs found in the monitored traffic, click the
browse button.
Replace
Type the domain name (real or fictitious) that you want to use instead of the one in
the monitored traffic. To select a domain name from a list of URLs found in the
monitored traffic, click the browse button.
Example
For example, the rule in the following image merges the data for all domains that match dell.*.com with the
data collected for dell.com.
Figure 59. Add Domain Rule dialog box
Storing large hit details
By default, Foglight stores up to 2 KB (2,048) characters for any field, header, or cookie value. For data values
that exceed this limit, you can configure an expanded storage (up to 64 KB) for specific hit details.
IMPORTANT: If you define some hit details as large, you also need to configure the Maximum Hit
Detail Size option in the Traffic Analysis Options view to match the size of the largest hit detail. For
more information, see Maximum Hit Detail Size in Modifying traffic analysis options.
To define large hit details, navigate to APM > Configure > Traffic Analysis, select Advanced Options, and
expand Large Hit Details.
Figure 60. Large Hit Details view
From the Large Hit Details view, you can add, copy, edit, and remove large hit detail definitions using the
standard controls.
For more information, see the following topics:
•
Defining large hit details
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
87
Defining large hit details
When adding, copying, or editing large hit details, you can set the following options.
Table 40. Defining large hit details
Option
Setting the Option
Name
Specify a unique name for the large hit detail definition.
Description
Optional—summarize the purpose for this definition.
Hit Detail Type
Select the detail type from the list: Request Header, Response Header, Request
Field, or Cookie.
Select an operator.
Operator
Hit Detail Name
TIP: To match multiple hit details with a single rule, select from operators such as
starts with, matches, or contains and specify the text to match in the Hit Detail
Name.
Specify a hit detail name, or text to use to match multiple details, or click the
browse button to select a hit detail from a list of details found in the live web
traffic.
Example
Figure 61. Add Large Hit Detail dialog box
Managing special events
Special events are global events—they can stop data collection for any in-progress session or sequence that
meets the conditions defined in the special event. In a fresh installation, special events are purposely
undefined. If desired, you can specify conditions required to trigger the special events. You can also specify
other actions to occur when a special event stops a session or sequence.
IMPORTANT: If someone else has defined special events for you, do not remove or edit these events until
you understand all implications of the change.
To manage special events, navigate to APM > Configure > Traffic Analysis, select Advanced Options, and
expand Special Events.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
88
Figure 62. Special Events view
From the Special Events view, you can edit the action performed by a special event. You can also remove a
special event using the standard control.
For more information, see the following topics:
•
Defining global stop criteria for sessions
•
Defining global stop criteria for sequences
•
Defining special events
Defining global stop criteria for sessions
While the Stop Session Event remains undefined, sessions can end in one of two ways: by a sequence analyzer
event’s Stop Session action or by a session timeout. The rules for determining when a session times out are set
in the sessionizing policy options. For more information, see Sessionizing policies—Defining when a session has
ended.
You can define the Stop Session Event to trigger on one or more event conditions. This enables you to stop data
collection on a session when something happens, such as an end user logs out, rather than waiting for the end
user’s session to time out.
As with all analyzer conditions, these event conditions can be grouped by the boolean operators AND and OR.
For more information, see Grouping multiple conditions.
For example, the following image shows a defined Stop Session Event triggered when an end user logs out.
Figure 63. Edit Special Event Page example
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
89
Defining global stop criteria for sequences
This event works much like the Stop Session Event, but rather than closing a session, it stops any active
sequences within a session. To consider activating this special event, you would need some event that is
common to all or many active sequences in your environment. For example, if you have multiple sequence
analyzers related to a shopping cart, you could define a Stop Sequence Event when an Empty Cart hit analyzer
matches.
Defining special events
When defining special events, you can set the following options.
Table 41. Defining special events
Option
Setting the Option
Description
Optional—summarize the conditions used in this event and the actions it performs.
You can add, edit, and group special event conditions in the same way as analyzer
conditions. For more information, see Configuring conditions.
Event Conditions
To add events, click Add. In the Add Event Condition dialog box, you can add event
conditions that are very similar to those available for sequence event conditions.
For a description of the types of conditions you can create, see Type option for
special event conditions (following this table).
Custom Fields
Optional—update the values of session-scoped custom fields. For more information,
see Updating custom fields.
Scripts
Optional—execute a script. For more information, see Executing scripts.
For examples, see Defining global stop criteria for sessions and Defining global stop criteria for sequences.
Type option for special event conditions
When defining event conditions for a special event, you can select any of the following values for the Type
option in the Add/Edit Event Condition dialog box.
NOTE: When reviewing the condition types, keep in mind the following:
•
Some types are listed as “Sequence/Session.” In the software, there are two separate condition
types, one for each.
•
For the “Hit Analyzer” conditions, you can optionally specify a hit status. To match the condition,
a hit must trigger the hit analyzer AND have the specified hit status.
Table 42. Type option for special event conditions
Type Option
Description
Hit Analyzer Match
Hits that trigger the specified hit analyzer also trigger the event.
Hit Analyzer Ordering
Requires two hit analyzers and an operator. The operator defines a relationship
between the two hit analyzers based on the timestamps of the hits that trigger the
hit analyzers. For example, you can create the following set of conditions to
compare the hit analyzers Foo and Bar within a given sequence: if Bar has triggered
AND if Foo has triggered AND if the Bar timestamp is less than the Foo timestamp,
then the event actions are triggered.
NOTE: If a hit analyzer matches more than one hit in a sequence, the hit analyzer
timestamp reflects the most recent match. This may lead to confusing results if you
were not expecting multiple matches.
Script Output
Requires a script, an operator, and an output value. The condition matches when
the script’s output value matches the value you specify.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
90
Table 42. Type option for special event conditions
Type Option
Description
Session Custom Field
Requires a session custom field, an operator, and a value. The condition matches
when the value of the custom field is updated to the specified value. The value may
be updated by a hit, sequence, or session analyzer.
Session/Sequence
Duration
Requires a metric, an operator, and a duration in seconds. The condition matches
when the metric’s value matches the specified duration. Duration is calculated from
the start of the sequence or from the first hit in the session.
Session/Sequence
Hit Counts
Requires the type of hits that should be counted, an operator, and the hit count.
The condition matches when the hit count for the sequence or session matches the
specified hit count. Hit count is calculated from the start of the sequence or
session.
Session/Sequence
Hit Counts by Hit
Analyzer
Requires a hit analyzer and a hit count for a given sequence or session. The
condition matches when the hit analyzer is triggered the specified number of times.
You can use this type of condition to identify recurring behavior. For example, you
can create a condition where hits that trigger the hit analyzer Failed Login Page are
counted. When the count reaches the specified hit count, let’s say five failed login
attempts, the event’s actions run.
Troubleshooting improperly classified
URLs
Foglight is typically able to identify which URLs represent the start of a new page download, and then from
these URLs it can correctly identify which URLs are hits that belong to the page. However, there may be cases
where Foglight is unable to correctly distinguish a page from a hit. For example, a URL is not recognized as the
starting point for a new page download, or a URL is incorrectly recognized as a new page, when it is actually an
element of the current page download. These situations typically arise with HTML frame-based applications and
with some rich internet applications. If you are not seeing accurate page metrics for some set of URLs, you
should add URL classifiers rules to classify the URLs.
To classify URLs, navigate to APM > Configure > Traffic Analysis, select Advanced Options, and expand URL
Classifiers.
Figure 64. URL Classifiers view
From the URL Classifiers view, you can add, copy, edit, and remove rules using the standard controls.
For more information, see the following topics:
•
Defining URL classifiers
Defining URL classifiers
When adding, copying, or editing URL classifier rules, you can set the following options.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
91
Table 43. Defining URL classifiers
Option
Setting the Option
Name
Specify a unique name for the rule.
Description
Optional—summarize the purpose for this URL classifier rule.
Class
Select a classification. The class determines whether hits on the specified URLs
should be treated as the start of a new page or as a hit within the current page. The
asynchronous classifications are useful with Rich Internet Applications (RIAs)—such
as those built using the AJAX framework—where pages do not need to be fully
downloaded before another page starts. See Class option (following this table).
Regular Expression
Specify a pattern to match against URLs in the monitored traffic. To test your
expression, click
to open the Extraction Rule dialog box. For more information,
see “Defining extraction expressions” in the online help.
Request Header
Optional—specify a request header that must also be present in the URL request
before this classification rule is applied.
Value
Optional—specify a value for the Request Header that must be present in the URL
request before this classification rule is applied.
Sessionizing Policy
When set to None (default), this rule applies to all URLs found in the monitored
traffic. There may be cases where it is advantageous to apply a rule only to the
URLs covered by a specific sessionizing policy. In this case, select the sessionizing
policy from this list.
Class option
You can select from the following classifications.
Class Option
Description
Starts a New
Page
Ends Current
Page Download
Is a Component
of Current Page
Generates
Page Metrics
Table 44. Class option
Page
Treats any hit on matching URLs as the start of a
new page.
Yes
Yes
No
Yes
Hit
Treats hits on matching URLs as hits on the
current page.
No
No
Yes
No
Asynchronous
Page
Treats any hit on matching URLs as a standalone
page.
Yes
No
No
Yes
Asynchronous
Hit
Treats a hit on matching URLs as a standalone hit
without attributing the hit to any page.
No
No
No
No
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
92
Example
Figure 65. Add URL Classifiers dialog box
Managing user agent rules
User agent rules transform strings found within the user-agent HTTP header into more meaningful strings for
display and categorization purposes. They can be used to rename the browser, browser category, and operating
system associated with a hit. Foglight applies the transformations before storing these items as hit details. For
definitions of hit details, see “Archiver Database Details” in the Foglight APM Reference Guide.
You only need to add user agent rules if the default categorizations do not suit your needs. For example, you see
a session where the browser, browser category, and operating system fields are empty. This usually indicates
that a newly-released browser is not yet categorized. You can categorize the new browser yourself by creating a
new user agent rule.
To review or upload user agent rules, navigate to APM > Configure > Traffic Analysis, select Advanced Options,
and expand User Agent Rules.
Figure 66. User Agent Rules view
In the User Agent Rules view, you can add, remove, and set the order for user agent rule files. These topics are
summarized in the following subsections. For procedures, see the online help.
In the User Agent Rules view, you can perform the following tasks:
•
Defining new user agent rules
•
Setting the execution order for rule files
Defining new user agent rules
You define new user agent rules in an XML file using the elements in the following table. Rules are orderdependent.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
93
CAUTION: Do not edit the Default Rules. This file is updated periodically, and you will lose your
changes.
Table 45. Defining new user agent rules
Element
Attribute
Description
Use this element to categorize an operating system.
osrule
find
The OS string contained in the user-agent header.
replace
The OS name you want to use for display and categorization
purposes.
Use this element to set the browser category, the browser name, and
the OS name.
useragent
browsercategory
A string that identifies the device used to create the hit, such as
Browser or Mobile Device.
find
A regular expression used to parse the user-agent header. For
more information, see Appendix: Building regular expressions in
Foglight.
ossource
Either a string or a token ($1, $2, etc.) matching one of the sets of
brackets in the regular expression.
browser
Either a string or a token ($1, $2, etc.) matching one of the sets of
brackets in the regular expression.
Example <osrule>
The following rules reassign two versions of Windows® NT to Windows® 8 for analysis and reporting purposes.
<osrule>
<find>Windows NT 7.0</find>
<replace>Windows 8</replace>
</osrule>
<osrule>
<find>Windows NT 6.2</find>
<replace>Windows 8</replace>
</osrule>
Example <useragent>
The following rule finds any version of Internet Explorer® Mobile and assigns it to the Mobile Device browser
category. The find attribute is defined as a regular expression that parses out the version numbers and the OS.
The operating system and browser hit details are dynamically assigned those values using tokens.
<useragent>
<browsercategory>Mobile Device</browsercategory>
<find>.*\(compatible; MSIE ([0-9]*)\.([0-9]*);.*(Windows Phone [0-9]*\.
[0-9]*).*</find>
<ossource>$3</ossource>
<browser>Internet Explorer Mobile $1.$2</browser>
</useragent>
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
94
Setting the execution order for rule files
User agent rules are order-dependent, both within a file and across multiple files. When two files transform the
same hit detail, the first file wins. When you finish adding new agent rules files, ensure that the order of the
files in the table reflects the order in which you want the rules applied. Use the Up
and Down
icons to
control the order of the other files.
Defining additional parsed response
types
For response content, Foglight captures and stores all response content, but only response content with a
content type containing one of the following patterns undergoes processing for traffic analysis:
•
text/.* (includes content types such as text/html, text/plain, and text/xml)
•
.*javascript.*
You can specify additional text-based content types for traffic analysis.
NOTE: Binary content types such as image, are automatically captured and stored, but are not processed
for traffic analysis.
Processing includes:
•
Parsing and storing response content
•
Indexing content for keyword search (if enabled)
•
Applying sensitive content rules. For more information, see Managing security policies for sensitive
content
•
Applying hit and sequence analyzers. For more information, see Configuring analyzers.
To define parsed response types, navigate to APM > Configure > Traffic Analysis, select Advanced Options, and
expand Additional Parsed Response Types.
Figure 67. Additional Parsed Response Types view
From the Additional Parsed Response Types view, you can add, edit, and remove content types using the
standard controls. When adding or editing response types to parse, specify the value of the content type in the
Name box.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
95
Example
Figure 68. Additional Parsed Response Types dialog box
Defining captured request body types
Request body content is treated a little differently than response content. Foglight captures and parses textbased POST data sent with either of the following content types:
•
application/x-www-form-urlencoded
•
multipart/form-data
The parsed content is stored as request fields. Unlike response content, the raw request body content is
discarded. These request fields are not keyword indexed or subjected to sensitive content rules, but are
available for traffic analysis purposes.
To define request body types for capture, navigate to APM > Configure > Traffic Analysis, select Advanced
Options, and expand Captured Request Body Types.
Figure 69. Captured Request Body Types view
From the Captured Request Body Types view, you can add, edit, and remove content types using the standard
controls. When adding or editing a capture request body type, specify a POST data content type (text-based
only) in the Name box.
Example
Figure 70. Captured Request Body Types dialog box
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
96
Modifying traffic analysis options
The traffic analysis options enable you to modify the maximum size for content capture and hit details. To
modify traffic analysis options, navigate to APM > Configure > Traffic Analysis, click Advanced Options, and
expand Traffic Analysis Options.
Figure 71. Traffic Analysis Options view
The following table describes all the options. Remember to save after making changes.
Table 46. Modifying traffic analysis options
Option
Setting the Option
Maximum Request Size
Specify the largest request to be captured. The maximum size is 1,048,576 Bytes (1
MB). Requests that exceed the specified size are truncated.
Maximum Response Size
Specify the largest response to be captured. The maximum size is 1,048,576 Bytes
(1 MB). Responses that exceed the specified size are truncated.
Specify the size of the largest hit detail to be captured. The maximum size is 65,535
bytes. Hit details that exceed the specified size are truncated. For more
information, see Storing large hit details.
Maximum Hit Detail Size NOTE: Web browsers may encode some characters before sending them to the web
server. Encoded characters require three bytes rather than one byte. If hit details
might contain encoded characters, specify a maximum size equal to the largest hit
detail size * 3. For example, the default value of 6,144 bytes was determined by
multiplying the default hit detail size of 2 KB (that is, 2048 bytes) by 3.
Managing scripts
Scripts are snippets of custom logic that extend the functionality of Foglight APM. You create scripts using the
Groovy scripting language. With scripts, you can perform the following tasks:
•
create sophisticated match conditions for analyzers
•
match patterns and extract data from traffic analysis details
•
run calculations
•
match URLs to classify URLs as types of pages or hits
•
expose the results of scripts as output variables, which in turn can be used as values for custom fields or
metrics
NOTE: Scripts for traffic analysis run on the Archiver database and directly affect data analysis and
storage activities. Foglight also offers a Script Console dashboard. You can use the Script Console to query
data stored in topology objects in the Foglight database.
To define and manage scripts, navigate to APM > Configure > Traffic Analysis, select Advanced Options, and
expand Scripts. You can also create scripts from hit analyzers, session analyzers, and sequence analyzer events;
the Scripts view is the same in all dashboards. In the Scripts dashboard, you can add, copy, edit, and remove
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
97
scripts using the standard controls. The Used By column contains a list of analyzers and events currently
configured to execute each script.
For more information, see the following topics:
•
Before you begin
•
Determining requirements
•
Creating scripts
•
Testing scripts
•
Defining scripts
Before you begin
To create scripts, you require a good working knowledge of the Groovy scripting language and Java™ regular
expressions. You also need to become familiar with the Traffic Analysis APIs, which are installed automatically
with the Foglight APM software.
NOTE: The Groovy scripting engine incorporates a security policy that limits access to certain classes and
methods. Most notably, it prevents access to the file system, system commands, threads, or processes.
Table 47. Reference information
Resource
Information
Javadoc for the Traffic Analysis For Javadoc on the Traffic Analysis APIs, see http://
APIs
<Foglight_IP_address>:8080/euwebgui/scriptdocs/index.htm.
Groovy scripting language
For general information about Groovy scripts, see
http://groovy.codehaus.org/Documentation.
Syntax for Java regular
expressions
For a primer on regular expressions, see Appendix: Building regular
expressions in Foglight.
Determining requirements
Before you begin adding scripts in the Scripts view, you should take the time to identify the key requirements
for your script. Then when you create the script, you know exactly which values to select in the Script - Add
dashboard and Edit Script dialog box.
Planning a script includes the following steps:
1
Select an appropriate API
2
Define output variables
3
Decide when to execute the script
4
Decide how many uncaught failures are allowed
Select an appropriate API
You have a choice of four Traffic Analysis API. When selecting an API, you need to consider:
•
the traffic analysis data your script requires
•
where you want to execute your script
For example, do you need only basic hit-level data, or do you require sequence-level detail such as the reason a
sequence stopped, or possibly session-level data such as the session hit count? Do you want to run your script
from a hit analyzer or a sequence analyzer or some other location?
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
98
Based on your data requirements, use the following table to select an appropriate API.
NOTE: Each API includes all the data available with the API that precedes it in the table. For example, the
SessionAnalysis API includes all hit details available in the HitAnalysis API. The APIs do not correspond to
the traffic analysis processing phases described in Appendix: Traffic analysis processing.
Table 48. Selecting an appropriate API
Data Requirements
Traffic Analysis API
Basic hit details, such as the URL, browser, response content, and headers.
For a complete list, see HitDetails in the Scripting API JavaDoc.
HitAnalysis
Basic session details, such as hit count and end to end time, plus extended hit
details, such as custom fields. For a complete list, see SessionDetails and
SessionAnalysis
HitDetailsExt in the Scripting API JavaDoc.
Also includes all data available with the HitAnalysis API.
Basic sequence details for the current sequence, such as pages in the
sequence and back end time. For a complete list, see SequenceDetails in the
Scripting API JavaDoc.
SequenceAnalysis
Also includes all data available with the SessionAnalysis API.
Extended sequence details such as stop reason and stop time. For a complete
list, see SequenceDetailsExt in the Scripting API JavaDoc.
PostSequenceAnalysis
Also includes all data available with the SequenceAnalysis API.
Generally speaking, as an API provides access to more data, the script execution options become more
restrictive. In other words, the more data you can access, the fewer places the script can run. For example, a
script that uses the HitAnalysis API can run anywhere a script is supported in the APM dashboards. On the other
hand, a script that uses the PostSequenceAnalysis API can be run only when the sequence analyzer event is in a
Stopped state.
Based on where you want to execute the script, use the following table to understand your options.
Table 49. Executing a script
Hit Analyzer
SequenceAnalysis
SessionAnalysis
HitAnalysis
To execute a script from a:
—
PostSequenceAnalysis
Select one of the following APIs:
—
—
Session Analyzer1
—
—
Sequence Analyzer Event
with Sequence Entry State=Not Started
—
—
Sequence Analyzer Event
with Sequence Entry State=Active
—
Sequence Analyzer Event
with Sequence Entry State=Stopped
Special Event—Stop Session Event
Special Event—Stop Sequence Event
—
—
—
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
99
1
When a timeout causes a session or sequence to stop, hit details are not available. If your script
attempts to access hit details, null values are returned. Therefore, whenever a script can be executed
due to a timeout, the script needs to be programmed either to avoid accessing hit details or to handle
the case where hit details are null.
Define output variables
Script output variables can be referenced in conditions, custom field updates, and metric updates. Decide how
many output variables your script requires and the type of value you want to assign to each output variable. The
value of an output variable can be a String, Integer, Long, Double, or Boolean value. You do not need to
set the data type explicitly, but you should decide now on what values you want to expose after the script runs.
Groovy often defaults to BigDecimal and BigInteger when it encounters numeric literals. In Foglight APM,
BigDecimal values are automatically converted to Double, and BigInteger is converted to Integer.
Decide when to execute the script
A script executes once in the selected context, such as once in the current hit or once in a session. Your choice
of API affects which execute options are available, as shown in the following table.
Table 50. Deciding when to execute a script
Traffic Analysis API
Once Per Hit
Once Per Session
Once Per Sequence
HitAnalysis
—
SessionAnalysis
—
SequenceAnalysis
PostSequenceAnalysis
When a script executes the first time in the selected context (hit/session/sequence), it caches the value of any
generated output variables. If other analyzers match the same hit/session/sequence and require the value of
the script’s output variables, they use the cached values; the script cannot execute again in the same hit/
session/sequence. Of course, if the hit analyzer never matches a hit, the script's output variable is not required
and therefore the script is not executed and the value for the output variable remains undefined (null).
For example, let’s say you create a HitAnalysis script with one output variable that is configured to run Once Per
Hit. A hit analyzer requires the script’s output variable to update a custom field. When the analyzer’s conditions
match a hit, the script executes exactly one time and caches the value of its output variable. When a second
analyzer matches the same hit, it can use the cached value, but cannot rerun the script.
IMPORTANT: For scripts with multiple output variables, any output variables whose values are not set
when the script executes the first (and only) time remain set to null.
If your script contains multiple output variables that are referenced either by the same analyzer or multiple
analyzers, you cannot control the order in which the script’s output variables are referenced. Any of the
references can cause the script to run. In this case, you need to write your script to generate values for all
output variables every time the script runs.
Decide how many uncaught failures are allowed
An uncaught failure is an uncaught exception that prevents the script from running to completion. Decide
whether you want to prevent the script from running after a certain number of uncaught failures. To ignore
script failures and continue to use the script, you can leave the default value of zero.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
100
Creating scripts
Now that you know your script requirements, you can begin coding your script. While you can code your script
directly in the Edit Script dialog box, if you intend to create a long or complex script, you may be more
comfortable drafting the script in your favorite text editor first. You can copy the draft script to the Edit Script
dialog box for syntax verification and sample data testing.
You can create scripts from the Traffic Analysis > Advanced Options dashboard or from any dashboard that offers
a Scripts view. From the Advanced Options dashboard, when you click Add in the Scripts view, the Script - Add
dashboard appears. From all other locations, when you click Add, you can choose to use an existing script or
create a new one. If you create a new one, the Script - Add dialog box appears.You specify the analysis API,
execute policy, uncaught failure policy, and output variables that you decided upon when determining your
script requirements. For more information, see Determining requirements and “Adding, copying, and editing
scripts” in the online help.
Let’s look at an example script. The following image shows the options required for a script that transforms
data. The HitAnalysis API is used for access to hit details and the script executes once per hit. Two output
variables are declared: productTotal and bigOrder. For more information, see Defining scripts.
Figure 72. Script—Add dialog box
In this dashboard, the Script Body box is display only. To add the body of the script, click Edit Script. From the
Edit Script dialog box you can type your script and validate the syntax.
NOTE: Foglight APM scripts are based on the Groovy scripting language. For a list of scripting resources,
see Before you begin.
Before reviewing the script for TransformData, you should be aware that all scripts have access to a default
binding containing the following two variables:
•
api—an instance of the API that you selected in the Type box, such as HitAnalysis. The api variable
provides access to traffic analysis data as well as to other key functionality such as regular expressions.
•
results—a collection used to store the values for the output variables (defined in the Output Variables
pane). Data generated by the script is put into the results collection through an assignment
statement, such as results.outputVariable = someValue.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
101
The following image shows the TransformData script. Behind the scenes, api is automatically bound to the
HitAnalysis API because Hit Analysis was selected in the Type box.
Figure 73. Edit Script dialog box
The script starts by getting the cookie associated with the hit. If the cookie has no value, the script exits
silently through the use of a return statement. Next, the script defines a pattern to match in the cookie and
then uses the API’s buildMatcher() method to do the work of matching the pattern in the current cookie. If
no match is found, the script exits silently. Otherwise, it sets the value of productTotal to the value found in
the cookie. If that value exceeds $1,000, the bigOrder variable is set to true. Both values are stored in the
default results collection.
This was a fairly straightforward example. Your scripts can be as simple or as complex as required to meet your
needs. When coding your own script, you can:
•
Reference any of the traffic analysis details (such as hit details and session details) exposed by the API
you selected. For a complete list, click the links in the Data Requirements table.
NOTE: Hit details are not available following a session or sequence timeout. If your script can be
executed immediately following a timeout, the script needs to be programmed so that either it
does not require hit details or it can handle the case where hit details are null.
•
Call any of the methods in your chosen API. For a complete list of classes and methods in the APIs, see
the corresponding list of interfaces in the API Javadoc.
•
Generate and cache values for the script’s output variables using the results collection. The values
can be used by analyzers in match conditions, custom fields, and metrics.
•
Use return statements to exit silently whenever a script cannot access data or meet some other
precondition it needs to proceed. When a script quits, any unset output variables continue to be
undefined (null) values. Exiting a script through a return statement does not count as an uncaught
failure.
•
Reference traffic analysis configuration objects, such as hit analyzers and custom fields. Because
configuration object names are not necessarily unique, you need to use the object’s UUID. You can look
up the UUID by clicking Lookup Object UUID. For more information, see “Referencing configuration
objects in a script” in the online help.
•
Add regular expressions to your script to do things like match a pattern. You can create and test regular
expressions in the Extraction Rule tool by clicking Regular Expression Tester. For more information, see
“Testing regular expressions” in the online help.
•
Check your syntax while you work by clicking Check Syntax. The Syntax Messages box displays errors or
warnings regarding basic syntactical errors. Security violations or references to undefined classes or
methods are not detected or reported.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
102
Testing scripts
While you create a script, you can check the syntax and test regular expressions. After you are done with the
script, you can run a test on the entire script using some sample data.
IMPORTANT: To ensure your scripts produce the results you expect, test scripts before using them.
In the Script - Add dashboard or dialog box, click Test Script. To run a test, add some sample data, specify the
expected results, and click Execute Test.
Each test data element consists of up to four parts. The general form of the test data is:
dataType.dataElement[#argumentValue ... ]=dataValue
where:
•
dataType is one of: hit, session, sessionHA, sequence or sequenceHA
•
dataElement is the specific piece of data you are specifying for that dataType
•
argumentValue is optional and consists of the arguments passed to the method used to access the
dataElement
•
dataValue is the value you wish to assign this dataElement.
For example, the test data for specifying the value of a hit custom field whose name is cartValue to a value
of 100.00 would be hit.customField#cartValue=100.00. For data elements that are retrieved by API
methods that do not take any arguments, simply omit the argumentValue. For example, the test data for
specifying the client IP would be hit.clientIP=10.2.2.1.
For a list of valid test data elements, click Help.
The following image shows the results of testing the TransformData script.
Figure 74. Test Script dialog box
Defining scripts
When adding, copying, or editing scripts, you can set the following options.
IMPORTANT: To be able to save a script, the script must be syntactically correct. If you need to save
before your script is finished, you can comment out any statements that contain syntax errors.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
103
Table 51. Defining scripts
Option
Setting the Option
Name
Specify a unique name for the script.
Description
Optional—summarize the actions perform by this script.
Type
Select the Traffic Analysis API that you want to use for this script based on your data
requirements and where the script will be executed. For more information, see
Select an appropriate API.
Execute
Select the context (hit/sequence/session) in which the script executes. Scripts
execute once for each matching hit/sequence/session. Any generated output
variable values are cached. For more information, see Decide when to execute the
script.
Maximum Number of
Uncaught Failures
Specify the number of uncaught failures that can occur before the script no longer
executes or use 0 to indicate unlimited failures. For more information, see Decide
how many uncaught failures are allowed.
Output Variables
Click Add. Specify a name for the output variable. You can create as many output
variables as your script requires. For more information, see Define output variables.
Script Body
Display Only—to add or edit a script, click Edit Script. For more information, see
Creating scripts.
Example
Figure 75. Script—Add dialog box
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring traffic analysis
104
4
Configuring analyzers
Analyzers are tools that you can use to collect additional data about hits and sessions, define and collect data
about sequences (a subset of hits or transactions within a session), annotate data, perform calculations, and
control data storage. When creating analyzers, you define a set of conditions and a series of actions. For
example, you can create an analyzer that matches every hit and, if the hit status is an error, the analyzer
increments an error counting metric and discards all other data about the hit.
Foglight offers three types of analyzers: hit, session, and sequence. This section introduces you to each type of
analyzer and describes how to create and use them. To get started with analyzers, select one of the following
topics:
•
Introducing hit analyzers
•
Introducing sequence analyzers
•
Introducing session analyzers
The following table lists common tasks and identifies the analyzer required to perform each task.
Table 52. Analyzer types
Task
Type of Analyzer
Monitor web application performance against
service-level agreements
Hit analyzer—Main phase
Monitor the performance of an important workflow
in your web application, such as a shopping to
checkout sequence
Sequence analyzer
View selected hits geographically in the Geographic Hit analyzer—Main phase
Perspective dashboard
Hit analyzer—Filtering phase
Discard hits that you do not care about, for
example, you may want to ignore hits generated by
crawlers and bots
Mark interesting hits so that you can retrieve them
later using a search by analyzer name
Hit analyzer—Main phase
Generate a set of standard metrics, create and
update your own metrics (custom metrics), or
generate metrics by category, such as operating
system or region (pivots)
Depends on the type of metrics required:
Annotate data with custom text (custom fields)
Depends. Custom fields have a scope, and in some cases,
more than one type of analyzer can update them. For
example, you could define a session-scoped custom field,
and then use one or more hit analyzers to set its value.
When sessions end, you may want to use the final value
of a session-scoped custom field as a match condition in
a session analyzer.
Extract hit details embedded in various types of
content and store the details (custom fields)
Hit analyzer—Main phase
•
hit-level: Hit analyzer—Main phase
•
session-level: Session analyzer
•
sequence-level: Sequence analyzer
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
105
Table 52. Analyzer types
Task
Type of Analyzer
Run your own calculations (scripts)
Depends on the data required:
Avoid storing sensitive data in the database
(security)
•
Hit-level: Hit analyzer—Main or Final phase
•
Session-level: Session analyzer
•
Sequence-level: Sequence analyzer
Hit analyzer—Main or Final phase
NOTE: Sessions and sequences may include hits that
contain sensitive data, but the sequence and session
objects themselves do not have sensitive data associated
with them.
For more details, see the following topics:
•
Summary of options by analyzer type
•
Configuring conditions
•
Working with metrics
•
Updating custom fields
•
Executing scripts
•
Defining storage policies
•
Setting searchability options for analyzers
•
Defining events for sequence analyzers
Introducing hit analyzers
A hit analyzer analyzes hits as they are captured. When a hit matches the analyzer’s conditions, the analyzer
actions run.
You can define hit analyzers to perform any of the following types of actions:
•
Monitor performance against service level agreements
•
Detect and alert on any per-hit condition
•
Mark interesting hits for later searching
•
Manage hit storage
•
Calculate performance measurements
•
Populate custom fields
•
Increment/decrement metrics
•
Extract data from hit details, using extraction expressions
•
Filter out hits you are not interested in
Foglight ships with a set of default hit analyzers to help you get started. Before creating your own hit analyzers
from scratch, you may want to review how the default analyzers are defined.
For more information, see the following topics:
•
Reviewing default hit analyzers
•
Tips for creating effective hit analyzers
•
Configuring hit analyzers
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
106
Reviewing default hit analyzers
This high-level workflow introduces you to hit analyzers by reviewing two default hit analyzers. You may want to
try these steps in Foglight.
1
On the navigation panel, under Dashboards, click APM > Configure > Traffic Analysis.
2
In the Traffic Analysis dashboard, expand Hit Analyzers.
You can add, copy, edit, and remove hit analyzers using the standard controls. You can also click Setup
Wizard to be guided through the process of creating a simple hit analyzer that matches on the request
path.
From the Enabled column, you can see that some of the default analyzers are enabled, which means the
Archiver evaluates their match conditions for captured hits. The disabled analyzers are ignored. In the
Phase column, you can see that each hit analyzer runs in one of three possible phases: Filtering, Main,
and Final (not shown).
For this walkthrough, we examine two analyzers, one that runs in the Filtering phase and one that runs
in the Main phase.
3
In the row containing the filter Default Type Exclusions, click Edit
.
The Edit Hit Analyzer dashboard appears.
4
Expand the Match Conditions view.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
107
First, we notice that this analyzer has multiple conditions, one per row. At the bottom left, you can see
that the Grouping for the conditions is a Boolean OR, so only one condition needs to be matched to
trigger this analyzer. For more information, see Configuring conditions.
These conditions match hits that would inflate the total hit count for a page, such as the hits required to
get the images for the page. If we enable this analyzer, all hits on the web server that retrieve
applications, images, CSS files, and JavaScript are excluded from data processing and storage. For
example, a hit to retrieve an image does not increment a hit count metric. In some environments, this
can be useful for keeping metrics meaningful.
A hit analyzer that runs in the Filtering phase has one implicit action, which is to drop any hits that
match the conditions. This means that the Sniffer does not process the hit and the hit is not sent it to the
Archiver. If you want to keep a hit and specify actions to perform on it, you need to specify a hit analyzer
that runs in the Main or Final phase. We look at a Main phase hit analyzer in the next steps.
TIP: You should consider using Filtering phase hit analyzers to reduce data processing and storage
for hits you are not interested in.
5
At the bottom of the dashboard, click Cancel to return to the Traffic Analysis dashboard.
Now let’s take a look at a Main phase hit analyzer. Most of the analyzers you create run in the Main
phase. This analyzer marks the first and last HTML pages viewed by a user during the user’s session.
6
In the row containing the filter Default Session Page Visit Custom Fields, click Edit
.
The Edit Hit Analyzer dashboard appears.
7
Expand the Match Conditions view.
This analyzer is triggered for hits on HTML pages. Below the match conditions, we can see a set of views
that represent actions that happen when a hit matches the condition. Some of the headings are
displayed in bold text, in this case, Metrics, Custom Fields, and Options. The bold text indicates that
the option has been defined or modified from its default behavior. For this workflow, we review all the
bolded views.
8
Expand Metrics.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
108
Notice that Generate Standard Metrics is deselected, which represents a change from the default
behavior for hit analyzers.
Metrics, and the ability to generate and update them in flexible ways, is a key concept in Foglight APM.
In some cases, like this one where the analyzer only annotates data, we do not need these metrics. In
most cases, you use analyzers to match interesting hits, and so you want to collect more data about
them. Standard metrics are a predefined set of metrics that includes metrics such as hit count and page
end-to-end time. You can set service-level thresholds for the metrics, generate metrics by pivot category
(resulting in metrics such as page end-to-end time by browser), and make metrics available to the toplevel Geographic Perspective dashboard. You can also create custom metrics. For more information, see
Working with metrics.
9
Expand Custom Fields.
This hit analyzer uses default, session-scoped custom fields to annotate the hit that starts a session and
the hit that ends a session. You can view the scope for a custom field by clicking Edit or you can see the
scope for all custom fields in the Custom Fields view on the Traffic Analysis dashboard. The Entry Page is
set on the first hit for a user session. Once it is set for the current session, it is not reset. The Last Page
Visited field is reset for every hit in a session. At the end of the session, this field contains the last page
viewed before the session ended.
10 Expand Options.
These options support searches based on an analyzer’s name. In this case, we are not interested in
searching by analyzer name, so Mark Hits is set to Never.
Marking a hit annotates the hit with the analyzer name. Searchable includes the analyzer name in the
Expert Search dialog box. For more information, see Setting searchability options for analyzers.
11 This completes the walkthrough. Click Cancel to avoid changing the default analyzer.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
109
Tips for creating effective hit analyzers
Keep in mind these tips for creating effective match conditions for hit analyzers:
•
Make the match conditions as specific as possible.
•
Avoid creating hit analyzers that match every hit, unless you truly want to perform an action on every
hit.
•
Whenever a hit analyzer (Foo) needs the value of a custom field that is set by another hit analyzer (Bar)
for the same hit, ensure that Bar runs in the Main phase and Foo runs in the Final phase. You cannot
control the order in which analyzers within the same phase execute.
•
Minimize the use of regular expressions in match conditions (especially with large hit details such as
Request Content or Response Content). Regular expression processing can negatively affect performance
when run on every hit.
•
Avoid using the Script Output option in match conditions. Executing a script on every hit can negatively
affect performance.
•
When creating multiple conditions, if a condition uses regular expressions or scripting you should add it
as the last condition. This allows simpler conditions to be evaluated first.
Configuring hit analyzers
When adding, copying, or editing a hit analyzer, you can define the following options.
NOTE: When editing a hit analyzer, click Properties to edit the Name, Description, and the Enabled
options.
Table 53. Configuring hit analyzers
Option
Setting the Option
Name
Specify a unique name.
Description
Optional—summarize the reason you created the hit analyzer, including the actions
it performs.
Enabled
Select this option to activate the analyzer as soon as you finish defining it.
Match Conditions
Add conditions that need to be met before the analyzer’s actions run. Using match
conditions, you can target the actions of an analyzer to specific types of web
traffic. For more information, see Configuring conditions.
Metrics
By default, the analyzer creates standard metrics for the hit. You can change the
default behavior, add custom metrics, change how topology objects are created and
named, define pivots, define SLA thresholds, and publish data for use in maps. For
more information, see Working with metrics.
Custom Fields
Optional—update the values of custom fields. For more information, see Updating
custom fields.
Scripts
Optional—execute a script. For more information, see Executing scripts.
Error Conditions
Optional—add conditions that need to be met before changing the status of the hit
to Error. For more information, see Defining error and warning conditions.
Warning Conditions
Optional—add conditions that need to be met before changing the status of the hit
to Warning. For more information, see Defining error and warning conditions.
Hit Storage Restrictions
By default, the Archiver stores all hits, regardless of the hit analyzer status. To
change this behavior, see Defining hit storage restrictions for hit analyzers.
Options
By default, hits that satisfy an analyzer’s match conditions are annotated with the
analyzer’s name to support searching by analyzer. To change this behavior, see
Setting searchability options for analyzers.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
110
Introducing sequence analyzers
A sequence is a series of hits or a set of transactions executed by an end user during their visit to a web site or
while using a web application. Any given user session may contain multiple sequences. You can capture
information about sequences by defining sequence analyzers.
A sequence analyzer identifies distinct sequences during active user sessions. You define sequence analyzers by
creating a set of sequence events, which usually includes a start sequence event, some intermediate events,
and a stop sequence event. Each event is evaluated independently. When an event’s conditions are satisfied,
the analyzer runs the actions defined for the event. For more information, see Defining events for sequence
analyzers.
NOTE: A key distinction between hit analyzers and sequence analyzers is that hit analyzers examine the
hit currently being processed, while sequence analyzers act on a series of hits within the active session.
Sequence analyzers are most commonly used to identify a logical sequence, such as the set of hits associated
with how a customer uses a shopping cart on a web site or the set of hits generated by an employee navigating
a critical workflow in your intranet application. However, you can also use sequence analyzers to detect and
alert on any activity within a session. For example, you can use a sequence analyzer to detect multiple events
that provide information about site problems and client activity within a related set of pages.
For more information, see the following topics:
•
Configuring sequence analyzers
Configuring sequence analyzers
When adding, copying, or editing a sequence analyzer, you can define the following options.
NOTE: When editing a sequence analyzer, click Properties to edit the Name, Description, Enabled, and
Enable any Disabled Hit Analyzers used by the Sequence Analyzer options.
Table 54. Configuring sequence analyzers
Option
Setting the Option
Name
Specify a unique name.
Description
Optional—summarize the reason you created the analyzer, including the event
actions it performs.
Enabled
Select this option to activate the analyzer as soon as you finish defining it.
Enable any Disabled Hit Select this option to activate any disabled hit analyzers that are required by the
Analyzers used by this
sequence analyzer’s events.
Sequence Analyzer
Events
Define events for this sequence. Each event can have its own conditions and
actions. If the events do not need to occur in order, select the Events not required
to be sequential check box below the Events table. For more information, see
Defining events for sequence analyzers.
Pivots
Optional—select categories for which you want to capture additional metrics. For
more information, see Understanding pivots for analyzer metrics.
Storage
By default, the analyzer always saves data associated with a sequence, events,
metric updates, and custom field updates. To change this behavior, see Defining
storage options for sequence analyzers.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
111
Introducing session analyzers
A session is a series of hits with the same session identifier that are submitted by the same browser client in the
same visit to the site. A session analyzer is a collection of conditions and actions used to analyze completed
sessions. Session analyzers give you a complete picture of the hits and sequences that occurred during a user
session, providing a valuable context for your data.
Unlike hit analyzers and sequence analyzers, which evaluate hits and sequences as data is captured, session
analyzers are evaluated after a session ends. A session ends when any of the following events occur:
•
A session times out
•
A sequence analyzer ends the session
•
A global Stop Session Event ends the session
Session analyzers are most powerful when used in conjunction with hit analyzers and/or sequence analyzers.
For example, you can create session-scoped custom fields that are updated by hit analyzers and sequence
analyzers, and then use the values of the custom fields as match conditions for a session analyzer.
For more information, see the following topics:
•
Tips for creating effective session analyzers
•
Configuring session analyzers
Tips for creating effective session analyzers
Keep in mind these tips for creating effective session analyzers:
•
Make the match conditions as specific as possible.
•
Avoid using the Script Output option in match conditions. Executing a script on all sessions can negatively
affect performance.
•
When creating multiple conditions, if a condition uses regular expressions or scripting you should add it
as the last condition. This allows simpler conditions to be evaluated first.
Configuring session analyzers
When adding, copying, or editing a session analyzer, you can define the following options.
NOTE: When editing a session analyzer, click Properties to edit the Name, Description, and Enabled
options.
Table 55. Configuring session analyzers
Option
Setting the Option
Name
Specify a unique name.
Description
Optional—summarize the reason you created the analyzer, including the action it
performs.
Enabled
Select this option to activate the analyzer as soon as you finish defining it.
Match Conditions
Add conditions that need to be met before the analyzer’s actions run. Using match
conditions, you can target the actions of an analyzer to specific types of web
traffic. For more information, see Configuring conditions.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
112
Table 55. Configuring session analyzers
Option
Setting the Option
Optional—add custom metrics, define pivots, and change how topology objects are
created and named. For more information, see Working with metrics.
Metrics
NOTE: Unlike hit analyzers, session analyzers always create standard metrics for
the session; there is no option to avoid generating them.
Optional—update the value of custom fields. For more information, see Updating
custom fields.
Custom Fields
Scripts
Optional—execute a script. For more information, see Executing scripts.
Error Conditions
Optional—add conditions that need to be met before changing the status of the
completed session to Error. For more information, see Defining error and warning
conditions.
Warning Conditions
Optional—add conditions that need to be met before changing the status of the
completed session to Warning. For more information, see Defining error and
warning conditions.
Options
By default, sessions that satisfy an analyzer’s match conditions are annotated with
the analyzer’s name to support searching by analyzer. To change this behavior, see
Setting searchability options for analyzers.
Summary of options by analyzer type
The following table summarizes the conditions and actions that you can set for each type of analyzer and
provides links to the topics in the rest of this section.
Table 56. Summary of options by analyzer type
Option
Hit
Analyzer
Sequence
Analyzer
Session
Analyzer
Match,
Error,
Warning
Event
Match,
Error,
Warning
Configuring conditions
Conditions
Yes
Yes (scoped to
analyzer or
event)
Yes
Working with metrics
Yes (scoped to
hit or session)
Yes (scoped to
sequence or
session)
Yes (scoped to
session)
Updating custom fields
Yes
Yes, using
events
Yes
Executing scripts
Yes
Yes
n/a
Defining storage policies
Metrics
Custom
Fields
Scripts
Storage
Policies
For help defining options, see:
and Defining event conditions
Options
Yes
n/a
Yes
Setting searchability options for analyzers
Event
Actions
n/a
Yes, using
events
n/a
Defining events for sequence analyzers
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
113
Configuring conditions
You define all types of conditions—match conditions, error conditions, warning conditions, and event
conditions—in the same way. You can add, edit, and remove conditions using the standard controls. When you
have multiple conditions, you can also group the conditions and create Boolean expressions with them.
NOTE: While you can use the same procedures to add, edit, and group event conditions, the types of
conditions you can define are unique to event conditions. Therefore, event conditions are discussed with
other event topics. For more information, see Defining event conditions.
For more information, see the following topics:
•
Defining match conditions
•
Defining error and warning conditions
•
Grouping multiple conditions
•
Defining conditions
Defining match conditions
Match conditions enable you to target the actions of an analyzer to specific types of web traffic. When the
conditions are met, the analyzer runs its actions.
IMPORTANT: It is possible to define an analyzer with no match conditions, in which case the analyzer
always matches. Be careful to avoid assigning time-consuming actions to these kinds of analyzers, or they
can adversely affect the performance of your monitoring system.
The Match Conditions view appears near the top of the Add Analyzer or Edit Analyzer view. For more
information, see Defining conditions.
Figure 76. Edit Hit Analyzer view
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
114
Defining error and warning conditions
For sessions, the completed session status is always OK unless you explicitly change it using an analyzer. For
hits, Foglight APM automatically sets the hit status to Warning or Error under the conditions outlined in the
following table.
Table 57. Defining error and warning conditions
Hit Status
Description
At least one of the following responses or exceptions occurs:
Error
Warning
•
HTTP Client Error—status code between 400 and 499, excluding 401.
•
HTTP Server Error—status code between 500 and 598.
•
Content Error—content is incomplete, possibly due to an application error or
network data loss.
•
Server Reset—the server sent a TCP reset. Some reasons for server resets include
unexpected behavior by the client or a server becomes overloaded.
•
Server Timeout—the server stopped sending content on the TCP connection. This is
usually caused by some kind of server overload situation.
•
Client Timeout—the client stopped acknowledging receipt of data from the server.
Two common reasons for client timeouts are that the client loses power or the client
becomes disconnected from the network.
Client Reset—the client sent a TCP reset. TCP resets are most often triggered by a real user
clicking the browser’s Back or Stop buttons or closing the browser.
You can override the default hit status assignment or set the completed session status by defining an analyzer’s
error or warning conditions. First, you define the match conditions for the analyzer. Then you use the Error
Conditions and/or Warning Conditions panes to specify the circumstances under which the status should be
changed to Error or Warning (respectively). For more information, see Defining conditions.
Figure 77. Warning Conditions view
Multiple analyzers can potentially evaluate the status of a hit or session differently. The rule in this case is
“worst status wins”. That is, if any analyzer sets the status to Error, the final status is Error. If one analyzer sets
the status to Warning and another sets it to Error, the final status is Error. There are situations where it may be
important to distinguish between the status according to analyzer X and the final status (after all analyzers
have evaluated the hit or session and set the status).
Example: How the status for a hit or completed session is resolved
Consider the error conditions and warning conditions for the following analyzers.
•
Analyzer A does not set the status.
•
Analyzer B sets the status to Warning.
•
Analyzer C sets the status to Error.
In this case, the final status is Error. However, the status according to the individual analyzers remains the
same: Analyzer A is OK; Analyzer B is Warning; and Analyzer C is Error.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
115
Grouping multiple conditions
If you have multiple conditions, you can arrange the conditions in the order you want them evaluated. You can
also create simple and complex Boolean expressions using the logical operators AND, OR, and NOT.
TIP: While sequence event conditions can be grouped, they rarely need to be grouped in this fashion.
OR example
You can join all conditions with the OR operator. For example, consider these conditions:
1
Request Path contains “shopping” OR
2
Value of the “cartValue” cookie is >0 OR
3
HTTP response code is >=500
Web traffic has to match only one condition for the analyzer actions to run. For more information, see
“Creating a simple Boolean expression” in the online help.
AND example
You can join all conditions with the AND operator. For example, consider these conditions:
1
Request Path contains “shopping” AND
2
Value of the “cartValue” cookie is >0 AND
3
HTTP response code is >=500
In this case, web traffic must satisfy all three conditions for the analyzer actions to run. For more information,
see “Creating a simple Boolean expression” in the online help.
Complex example
When you define at least three conditions, you can choose to create an expression that uses more than one kind
of logical operator. For example, given the following conditions:
1
Request Path contains “shopping”
2
Value of the “cartValue” cookie is >0
3
HTTP response code is >=500
You can create complex expressions to express the following kinds of relationships:
•
1 matches AND either 2 OR 3 matches
•
1 matches OR both 2 AND 3 match
•
1 AND 2 match but NOT 3
Any valid combination is possible. For more information, see “Creating a complex Boolean expression” in the
online help.
Defining conditions
When adding or editing conditions, you can set the following options.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
116
Table 58. Defining conditions
Option
Setting the Option
Source
Select the type of web traffic matched by this condition. For example, Request
Path, Request Field, Browser, Cookie, and so on.
NOTE: The other available fields change depending on the selected Source.
Source Detail
If the Source you selected requires additional detail, specify the detail in this box.
For example, if you chose Source=Cookie, type the name of the cookie you want to
reference.
not
If available, select to cause the Operator to perform its opposite action. For
example, you can change equal to to not equal to by selecting this check box.
Operator
If available, select an operator that performs the type of match you want to make.
Only the operators that make sense for a selected Source type are available. For
more information, see Operators.
Value
If available, specify a numeric or text value (as appropriate). For example, you can
specify that a match occurs when Client Time (ms) is greater than 600. Or, for
Country, you can select a text string from a list, such as “Canada (CA)”.
Extraction Expression
Optional—if available, you can create an extraction expression. For more
information, see “Defining extraction expressions” in the online help.
Operators
The Operators list includes only the operators that make sense for the selected Source type.You can cause an
operator to perform its opposite behavior by selecting the not option. The following table contains all the
possible operators.
Table 59. Complete list of operators
Operator
=
Matches when a traffic element...
Equals the value in the Value field.
1
Starts With1
Starts with the value in the Value field.
Ends With1
Ends with the value in the Value field.
Contains1
Includes the value in the Value field.
Matches
Matches a condition specified in a regular expression.
<
Is less than the value in the Value field.
>
Is greater than the value in the Value field.
<=
Is less than or equal to the value in the Value field.
>=
Is greater than or equal to the value in the Value field.
Exists
Exists, that is, the specified Source/Source Detail occurs in monitored traffic.
1
You can select a version of this operator that ignores case.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
117
Example
Figure 78. Add Match Condition dialog box
Working with metrics
Foglight APM supports different groups of metrics:
•
Standard metrics—Metrics such as Hits Captured, Back End Time, and End to End Time, generated
when a hit matches an analyzer’s match conditions. For more information, see the Foglight APM
Reference Guide.
•
Custom metrics—User-defined metrics that can be created and updated using analyzers.
•
Pivot metrics—Metrics by category, which means that an Archiver stores the same set of metrics for
different segments of users defined by, for example, the browser they use or the operating system of the
device they use to access the monitored applications. Pivots apply to standard metrics and custom
metrics.
Because analyzers serve different purposes, each type of analyzer organizes the options for metrics slightly
differently. For example, while hit and session analyzers control pivots in the Metrics view, sequence analyzers
control pivots at the analyzer level and offer two kinds of metrics (sequence metrics and sequence event
metrics) at the event level. In all cases, however, you can define custom metrics, define the name of the
topology objects created (the name used for storage purposes), and specify pivots. For hit analyzers only, you
can also choose not to generate standard metrics, specify SLA thresholds, and make metrics available for
mapping purposes. The following image shows a hit analyzer with all the options available.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
118
Figure 79. Metrics view
For more information, see the following topics:
•
Understanding standard metrics
•
Creating custom metrics
•
Changing the name of metric topology objects
•
Understanding pivots for analyzer metrics
•
Publishing hit metrics for use in the Geographical Perspective dashboard
•
Defining service level agreement thresholds
Understanding standard metrics
Standard metrics are a set of metrics suitable for analyzing data about hits, sequences, and sessions. Standard
metrics are updated whenever the match conditions defined for an analyzer or sequence event are satisfied.
For lists of standard metrics, see the Foglight APM Reference Guide.
For hit analyzers only, you have the option of turning off standard metrics. For example, if an analyzer is
designed to match a failed login attempt, you may be interested in a counting the failed attempt (using a
custom metric), but you may not want to generate and save all the standard hit metrics. In this case, you can
clear the Generate Standard Metrics option.
Creating custom metrics
Custom metrics are user-defined, numeric variables. If an existing metric does not satisfy your requirements,
you can create a custom metric. For example, a typical reason to create a custom metric is to count some kind
of error. For more information, see Defining custom metrics.
You do not set explicit values for custom metrics, instead you use analyzers to increment or decrement the
value. An analyzer can update custom metrics using any of the following values:
•
Fixed values—For example, counter-type metrics are usually incremented by 1.
•
Script outputs—Analyzers can update metrics using values that are generated by Groovy scripts and
exposed as output variables. The script is executed as needed to determine the metric increment value.
•
Session custom fields—Analyzers can update metrics using numeric data from a session custom field.
•
Hit details (numeric)—Analyzers can update metrics using any numeric data associated with a hit. For
example, you can update a metric with Request Field values or Cookie values.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
119
Updating custom metric values
For custom metrics, you can define when the metrics are updated. By default, analyzers update the metrics
when the analyzer match conditions are satisfied or, in the case of sequence events, when the sequence stops.
You can update a custom metric on a different schedule by changing its Increment policy. The following table
describes the behavior of the Increment options, where <status> can be OK, Error, Warning, or a combination
thereof.
Table 60. Updating custom metric values
Analyzer
Always Update (default)
Other Options
Hit
If the hit satisfies the match
conditions, the custom metric is
updated immediately.
You can update a custom metric based on the status of
the analyzer or hit:
•
final hit status = <status>
•
hit analyzer status = <status>
Sequence
At the end of the sequence, if the
sequence satisfies the event’s match
conditions, the analyzer-scoped
custom metric is updated.
You can update an analyzer-scoped custom metric based
on the <status> of the analyzer.
Sequence
Event
At the end of a sequence, if the
sequence satisfies the event’s match
conditions, the event-scoped custom
metric is updated.
You can update an event-scoped custom metric based on
the <status> of the analyzer.
Session
At the end of a session, if the session You can update a custom metric based on the status of
satisfies the match conditions, the
the analyzer or session:
custom metric is updated.
• final session status = <status>
•
session analyzer status = <status>
Defining custom metrics
When adding or editing custom metrics, you can set the following options.
Table 61. Defining custom metrics
Option
Setting the Option
Name
Specify a unique name for the custom metric.
Description
Optional—summarize the purpose of this custom metric.
Source
Select the source of the value used to increment this metric—such as Fixed Value,
Script Output, Session Custom Field, or one of the available details.
NOTE: The other available fields change depending on the selected Source.
Source Detail
If the Source you selected requires additional detail, specify the detail in this box.
For example, if you selected Fixed Value, type the amount by which you want to
increment/decrement the metric.
Extraction Expression
Optional—if available, you can create a user-defined expression that extracts a
portion of the source’s value. For example, if your source contained “$99,” you can
extract “99” as your increment value. For more information, see “Defining
extraction expressions” in the online help.
Increment
By default, a custom metric is always incremented. You can change the update
policy. For more information, see Updating custom metric values.
For an example, see “Adding custom metrics” in the online help.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
120
Changing the name of metric topology objects
By default, Foglight stores all metrics in a topology object with the same name as the analyzer. You can change
the name of the topology object. You can also break out the metrics into multiple, dynamically-named topology
objects using a name substitution token.
Renaming a topology object
You may want to rename a topology object if you have multiple analyzers with similar names. You can make
your topology objects easier to locate by adding your name to the object name or by including some other
identifying text. To rename a topology object, select Customize Name and type the new name in the associated
field.
Creating dynamically-named topology objects
Foglight automatically creates topology objects to track hits that return HTTP error codes. However, you could
achieve the same result using dynamically-named topology objects. In this example, an analyzer matches hits
when the Response Code is 400 or greater than 401. By using Topology Object Naming, this one analyzer can
create objects for each error code and collect data separately.
In the Metrics view, in the Topology Object Naming pane, this analyzer has a customized name that includes the
$1 token. The $1 token is defined as a Response Code, which contains the error code.
Figure 80. Edit Hit Analyzer view
As this analyzer matches hits in the monitored traffic, new topology objects are created dynamically for each
error code encountered in the hits. For example,
•
HTTP Error Code 400
•
HTTP Error Code 403
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
121
•
HTTP Error Code 404
•
and so on
The topology objects created by this default analyzer appear in the Data dashboard. In the navigation panel,
under Dashboards, navigate to Configuration > Data. Expand APM > Transactions.
Figure 81. Configuration > Data > APM > Transactions view
When you select a topology object, you can see its list of properties and the Property Viewer opens to the right
of the list, as shown in the following image. The standard and custom metrics are organized under
Measurements. Under Pivots, you can find standard and custom metrics by pivot category.
Figure 82. Pivots category
For instructions, see “Breaking out metrics into dynamically-named topology objects” in the online help.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
122
Understanding pivots for analyzer metrics
With pivots, you can collect additional data based on a category, such as Country. Each item in a category
generates a new set of metrics (standard metrics plus custom metrics) and associated data. If you have
dynamically-named topology objects, each object includes the pivots. For a list of pivot categories, see
Understanding pivot categories.
IMPORTANT: Foglight APM contributes only a portion of all data handled by Foglight. To avoid flooding
Foglight with unnecessary data, avoid selecting all pivot categories for analyzers that fire frequently. As a
rule of thumb, collect just the data you need.
For example, if you select Country as a pivot category for logins, and people from two different countries visit
your site, you get a total of three sets of metrics, the basic set and one for each country.
Example of pivots for analyzer metrics
Login
------------------------------------------------Hits Back End Time
Hits Back End Time Exceeds Threshold
Hits Back End Time Service Level
Hits Captured
Login from China
Hits Captured Rate
-----------------------------------------------Hits Client Time
Login from India
Hits Back End Time
Hits Encrypted
-----------------------------------------------Hits Back End Time Exceeds Threshold
Hits Encrypted Percent
Hits Back End Time Service Level Hits Back End Time
Hits End to End Time
Hits Back End Time Exceeds Threshold
Hits Captured
:
Hits Back End Time Service Level
Hits Captured Rate
:
Hits Captured
Hits Client Time
Hits Captured Rate
Hits Encrypted
Hits Client Time
Hits Encrypted Percent
Hits Encrypted
Hits End to End Time
Hits Encrypted Percent
:
Hits End to End Time
:
:
:
To select pivots for an analyzer, click Pivots and select the pivot categories. For detailed instructions, see
“Selecting pivot categories” in the online help.
IMPORTANT: The analyzer-level pivot categories inherit settings from the Traffic Analysis > Pivots
dashboard. Although you may be able to select a pivot category for an analyzer, if that pivot category
does not have any values defined on the Traffic Analysis > Pivots > Pivot Objects dashboard, the
analyzer cannot generate the pivot metrics. For more information, see Managing pivots.
Example: Investigating a problem report using pivots
You have a hit analyzer, called Login, that tracks when end users access their web site login page. Standard
metrics only are collected—no pivot categories, no custom metrics.
An end user reports having difficulties accessing the login page using BrowserA version 1.1, but has no problem
using BrowserB. In Foglight, you update the Login hit analyzer to gather more data. Because you do not know if
any other browsers or browser versions are also affected, you enable the Browser category as a pivot. From the
resulting data, you can see when and where the errors occur. You may discover that the problem affects only a
few versions of BrowserA, and you can focus your efforts to resolve the problem on those browsers.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
123
Publishing hit metrics for use in the
Geographical Perspective dashboard
Foglight users with the role of APM Operator use the Geographical Perspective dashboard to understand how
monitored web sites and web applications are performing by country, region, and/or city. As the APM
Administrator, you control which metrics are available to the Geographical Perspective dashboard by enabling
map-based metric snapshots on an analyzer-by-analyzer basis. You choose the level of geographical detail you
want to display on the Geographical Perspective dashboard—that is, data points by country, region, and/or
city—and decide how many data points to plot at each level, from 0 to 100. For example, you could enable the
top 10 countries and the top 100 cities. For detailed instructions, see “Publishing metric timeslices for the
Geographical Perspectives dashboard” in the online help.
Figure 83. Maps dialog box
While the hit analyzer is active, it publishes snapshots of its metric data every five minutes. In the Geographical
Perspective dashboard, you select a measurement (metric) and a five-minute timeslice, and the dashboard
displays metric data from the appropriate snapshot. For more information, navigate to APM > Geographical
Perspective, select the Help tab, and click More.
Defining service level agreement thresholds
Service level agreements (SLA) define quality goals for delivery of service. You use hit analyzers to specify SLA
thresholds. For every hit analyzer you create, a set of default SLA thresholds are also created. You can change
these defaults to match your SLAs.
You can specify thresholds for key performance metrics and navigation time metrics (in milliseconds) as
described in the following table. The maximum value is 5 minutes (300000 ms). To disable a threshold, specify
zero (0). For detailed instructions, see “Changing SLA thresholds” in the online help.
Table 62. Defining service level agreement thresholds
SLA Threshold
Description
Hits Back End Time
Maximum amount of time that web servers should spend processing hit requests
sent from client browsers.
Hits End to End Time
Maximum amount of time that clients should experience when downloading a hit.
Pages Back End Time
Maximum amount of time that web servers should spend processing page requests
sent from client browsers.
Pages End to End Time
Maximum amount of time that clients should experience when downloading a
page.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
124
Table 62. Defining service level agreement thresholds
SLA Threshold
Description
Navigation Timing1
Content Delivery Time
Maximum amount of time that should elapse from the start of the HTTP request
to the start of the HTTP response, as measured using browser-based
instrumentation and following the W3C Navigation Timing standard.
1
Navigation Timing Page
Completion Time
Maximum amount time of time that should elapse from the start of a navigation
request (such as a user click on a hyperlink in a web page) to the end of the load
event, as measured using browser-based instrumentation and following the W3C
Navigation Timing standard. This is also sometimes referred to as “click to glass”
time.
1
To enable the navigation timing thresholds, you need to instrument the web application or site that you
want to monitor. For more information, see Capturing traffic outside the network.
Example
Figure 84. SLA Thresholds dialog box
Updating custom fields
Custom fields are useful for associating additional information with individual hits, sequences, or sessions. For
more information, see Managing custom fields.
Figure 85. Custom Fields view
You can update the value of custom fields using analyzer actions or special event actions. You can add, edit, and
remove custom field updates using the standard controls.
For more information, see the following topics:
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
125
•
Updating the value of custom fields
•
Creating new custom fields
•
Defining custom field updates
Updating the value of custom fields
A custom field update requires two things: the name of a properly scoped custom field and the source of the
value that you want to use to update the custom field. Different types of analyzers can update custom fields
with different scopes, as summarized in the following table. For more information, see “Adding custom field
updates” in the online help.
NOTE: You can update custom fields scoped for hits or sessions using more than one analyzer. For
sequence-scoped custom fields, only the analyzer that defines the custom field can update it.
Table 63. Updating the value of custom fields
Analyzer/Event
Custom Field Scope
You can update custom fields with the following scopes:
Hit
•
Hit scope
•
Session scope (for the session that contains the hit)
You can update custom fields with the following scopes:
Sequence (Event)
•
Sequence-analyzer scope (only the analyzer that defines the custom field
can update it)
•
Session scope (applies to the session that contains the sequence)
Session
Session scope
Special Event
Session scope
An analyzer can update custom fields using any of the following value types:
•
Fixed values—You can replace the value of a custom field with a new value or increment/decrement an
existing value.
•
Script outputs—Analyzers can use any data that is exposed as an output variable by a Groovy script. The
script is executed as needed.
•
Session custom fields—Analyzers can update custom fields using the data from a session-scoped custom
field.
•
Details—Analyzers can update custom fields using data that is already associated with a hit, sequence, or
session. For example, you can update a custom field with a Request Header value. The type of details
available to an analyzer changes depending on the type of analyzer.
A special event can use fixed values, script outputs, and session-scoped custom fields.
By default, analyzers update custom fields whenever the analyzer’s actions run. For hit and session analyzers,
you can specify the update criteria for each individual custom field update. For sequence analyzers, all custom
fields are updated at the same time. For more information, see Update criteria option.
Creating new custom fields
If an existing custom field does not suit your purposes, you can create a new custom field on the fly. In the
Custom Field Wizard, select Create New Custom Field. For more information, see Managing custom fields and
see “Adding custom field updates” in the online help.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
126
Defining custom field updates
When adding or editing custom field updates, you can set the following options.
Table 64. Defining custom field updates
Option
Setting the Option
Custom Field
Select a custom field or click New to create a custom field. For more information,
see Creating new custom fields.
Select the type of update, such as Fixed Value, Script Output, Session Custom Field,
or one of the available details.
Source
NOTE: The other fields displayed in this pane depend on this selection.
Source Detail
If the Source you selected requires additional detail, specify the detail in this box.
For example, if you selected Primary Value=Fixed Value, type the amount by which
you want to increment/decrement the custom field.
Extraction Expression
Optional—if available, you can create an extraction expression. For more
information, see “Defining extraction expressions” in the online help.
Update If Blank
Optional—override the setting defined for the custom field. Select to allow the
custom field to be set to an empty (blank) value. Clear (default) to ignore blank
values, that is, an existing value is not replaced by a new value if the new value is
blank.
Value Assignment
Optional—override the setting defined for the custom field. Select an option that
reflects how you want to handle values for the custom field. For more information,
see Value assignment mode option.
Separator
Optional—override the setting defined for the custom field. When the Value
Assignment Mode is set to one of the Append options, you can define a separator to
be inserted between values. Typical separators are a comma or a colon, but you can
use any character or combination of characters that suits your needs.
Update
Select the update criteria. See Update criteria option.
Update criteria option
The following table describes the possible values for the Update Criteria option, where <status> can be OK,
Error, Warning, or a combination thereof.
Table 65. Update criteria option
Analyzer
Always (default)
Other Options
Hit
Updates custom fields if a hit satisfies the You can choose to update based on:
match conditions.
• final hit status = <status>
•
hit analyzer status = <status>
Sequence
Commits pending custom field updates if
the sequence stops.
You can commit pending updates based on:
You can choose to update based on:
Session
Updates custom fields if a session
satisfies the match conditions.
•
sequence analyzer status = <status>
•
final session status = <status>
•
session analyzer status = <status>
Executing scripts
Scripts are snippets of custom logic written in the Groovy scripting language. You can create scripts to extend
the functionality of Foglight APM Appliances. For more information, see Managing scripts.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
127
NOTE: Scripts for traffic analysis run on the Archiver database and directly affect data analysis and
storage activities. Foglight also offers a Script Console dashboard. You can use the Script Console to query
data stored in topology objects in the Foglight database.
Any type of analyzer (or special event) can execute a script. Analyzers can use the output variables published by
the script to update custom fields or metrics.
Figure 86. Scripts view
You can add, edit, and remove scripts from an analyzer using the standard controls. For more information, see
the following topics:
•
Executing scripts
•
Creating new scripts
Executing scripts
To execute a script from an analyzer, you specify the name of a validated script and the execution criteria. By
default, analyzers execute scripts whenever the analyzer’s conditions are satisfied. For hit and session
analyzers, you can choose a different criteria for executing scripts.
The following table describes the behavior of the Executing Criteria options, where <status> can be OK, Error,
Warning, or a combination thereof.
Table 66. Executing scripts
Analyzer
Always (default)
Other Options
You can choose to execute scripts based on:
Hit
Executes scripts if a hit satisfies the
match conditions.
Sequence
Session
•
final hit status = <status> OR
•
hit analyzer status = <status>
Executes scripts if the event conditions
are satisfied.
Not applicable.
Executes scripts if a session satisfies the
match conditions.
You can choose to execute scripts based on:
•
final session status = <status> OR
•
session analyzer status = <status>
Creating new scripts
If an existing script does not suit your purposes, you can create a new script on the fly. You create the script in
exactly the same way you would if you selected Scripts from the Advanced Options. For more information, see
Managing scripts and “Creating new scripts on the fly” in the online help.
Scripts are global objects. If you create a new script while defining your analyzer’s actions, the script is
available to all analyzers that can execute the same type of script. For more information on types of script, see
Select an appropriate API.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
128
Defining storage policies
For hit analyzers and sequence analyzers, you can set the storage policies for whether data is stored, when it is
stored, and for hit analyzers, how much of the hit detail is stored.
For more information, see the following topics:
•
Defining hit storage restrictions for hit analyzers
•
Defining storage options for sequence analyzers
Defining hit storage restrictions for hit
analyzers
The default storage policy is to store all hits, regardless of the hit analyzer status. You can define one or more
hit storage restrictions to restrict the type of data stored based on the status of the hit analyzer. In this way,
you can tailor your storage to make the best use of the available disk space. When a hit matches multiple hit
analyzers, any overrides to the default storage policy are honored.
Figure 87. Hit Storage Restrictions view
When adding or editing hit storage restrictions, you can set the following options.
Table 67. Hit storage restrictions
Option
Setting the Option
Do not store hit
Select (default) to avoid storing data for the hits that match the hit analyzer’s
conditions. Clear to store hit details.
Do not store request or
response content
Select (default) to avoid storing response content for the hits that match the hit
analyzer’s conditions. Clear to store the response content.
Index response content
Select to index the response content and make the hit’s content keywordsearchable. Clear (default) to avoid indexing stored response content.
Do not store hit details
marked “Always
Sensitive”
Select to apply any Sensitive Hit Detail rules defined as “Always Sensitive” before
storing a hit. This option enables you to avoid storing sensitive fields, headers, or
cookie values. Clear (default) to store all hit details. For more information, see
Managing security policies for sensitive hit details.
Do not store request or
response content
marked “Always
Sensitive”
Select to apply any Sensitive Response Content Expressions rules defined as “Always
Sensitive” before storing a hit. This option enables you to hide sensitive data
contained in the response content before storing the hit. Clear (default) to store all
response content. For more information, see Managing security policies for
sensitive content.
Update
Set the criteria under which the hit analyzer applies the storage policy.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
129
Example
Figure 88. Add Hit Storage Restriction dialog box
Defining storage options for sequence analyzers
For sequence analyzers, you can set separate storage policies for the sequence itself and for its events. By
default, the Archiver always stores data associated with these elements.
Figure 89. Defining storage options for sequence analyzers
For each option, select the value that matches your storage requirements:
•
Always—Default. Data is always stored.
•
Never—Data is never stored.
•
Conditional when sequence analyzer status is—Data is stored only when the status of the sequence
analyzer matches the status you select in the list.
When selecting options, you may find the following information about the options helpful.
Setting searchability options for
analyzers
Analyzer options affect the searchability of your data by analyzer name. In particular, the options you select
here affect the ability of the Expert Search feature to run a search based on an analyzer. For more information,
see the “Defining expert searches” topics in the Foglight APM User Guide online help.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
130
Figure 90. Setting searchability options for analyzers
By default, when a hit or session satisfies an analyzer’s match conditions, the analyzer annotates (or marks) the
hit or session with the name of the analyzer. When you search your captured data based on an analyzer’s name,
you see the hits or sessions that were marked. In some circumstances, you may not need to annotate your data
with the analyzer name, or you may want to annotate based on the status of the hit or session. For more
information, see Defining options for hit analyzers and Defining options for session analyzers.
The Searchable option causes the name of the analyzer to be included in the list of analyzers contained in the
Expert Search dialog box. If you do not want an analyzer to be used for searches, you should clear this option.
For more information, see the following topics:
•
Defining options for hit analyzers
•
Defining options for session analyzers
Defining options for hit analyzers
You can set the following options.
Table 68. Defining options for hit analyzers
Option
Setting the Option
Mark Hits
By default, all hits that triggered the hit analyzer are always annotated with the
analyzer’s name. You can choose to never mark hits or mark them conditionally
based on the hit status. See Mark Hits option.
Searchable
Select (default) to include the analyzer name in the Hit Analyzer list in the Expert
Search dialog box. Clear to exclude the analyzer from the list.
TIP: Consider clearing this option when the analyzer matches every hit.
Mark Hits option
For hit analyzers, you can select any of the following values for the Mark Hits option.
Table 69. Mark Hits option
Mark Hits Option
Description (for Hit Analyzers)
Always
Always annotates hits with the name of the hit analyzer.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
131
Table 69. Mark Hits option
Mark Hits Option
Description (for Hit Analyzers)
Never annotates hits. The connection between the hit and the analyzer is lost.
Searches based on the hit analyzer name return no results.
Consider selecting this option when:
Never
•
A hit analyzer only supports sequence or session analyzers.
•
A hit analyzer only manages hit storage.
•
A hit analyzers matches every hit. In this case, a search by analyzer name
would return the same results as a search for all hits. Annotation provides no
additional value.
Set the criteria under which the hit analyzer annotates a hit. You can mark hits
based on:
Conditional
•
final hit status = <status> OR
•
hit analyzer status = <status>
where <status> is OK, Error, Warning, or a combination thereof.
Defining options for session analyzers
You can set the following options.
Table 70. Defining options for session analyzers
Option
Setting the Option
Mark Sessions
By default, all sessions that triggered the session analyzer are always annotated
with the analyzer’s name. You can choose to never mark sessions or mark them
conditionally based on the session status. See Mark Sessions option.
Searchable
Select (default) to include the analyzer name in the Session Analyzer list in the
Expert Search dialog box. Clear to exclude the analyzer from the list.
TIP: Consider clearing this option when the analyzer matches every session.
Mark Sessions option
For session analyzers, you can select any of the following options for the Mark Sessions option.
Table 71. Mark Sessions option
Mark Sessions Option
Description (for Session Analyzers)
Always
Always annotates sessions with the name of the session analyzer.
Never annotates sessions. The connection between the session and the analyzer is
lost. Searches based on the session analyzer name return no results.
Consider selecting this option when:
Never
•
A session analyzer updates only metrics.
•
A session analyzer updates only custom fields.
•
A session analyzer matches every session. In this case, a search by analyzer
name would return the same results as a search for all sessions. Annotation
provides no additional value.
Set the criteria under which the session analyzer annotates a session. You can mark
sessions when:
Conditional
•
final session status = <status> OR
•
session analyzer status = <status>
where <status> is OK, Error, Warning, or a combination thereof.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
132
Defining events for sequence analyzers
Sequence analyzers usually have multiple events, each with their own conditions and actions.
Figure 91. Events view
You can use events to perform the following tasks:
•
Start sequences
•
Change the status of the sequence
•
Update metrics
•
Update custom fields
•
Run scripts
•
Stop sequences
•
Stop sessions
Events are dependent on the state of the sequence, which means that event conditions will be evaluated only
when the sequence is in the specified entry state.
For more information, see the following topics:
•
Understanding sequence states
•
Defining event conditions
•
Recording the order of triggered events
•
Defining sequence events
Understanding sequence states
A complete sequence flows through three states: Not Started, Active, and Stopped. When you define events for
your sequence, you specify the sequence state that must exist before the event conditions are evaluated.
For example, let’s say that you want to start a sequence for the first time in a session. You need to create an
event with at least the following settings:
•
Sequence Entry State=Not Started
•
Event Actions: Start/Stop Actions=Start Sequence
In this case, the event condition will be evaluated when the sequence state is Not Started. The Not Started
state indicates that no sequences defined by this sequence analyzer have started for the current session. This
event’s only action is to start the sequence, which sets the sequence state to Active.
The following table summarizes the effect of stop/start actions on the sequence state. When you stop a session,
the sequence also stops because sequences cannot extend beyond the end of the current session. Note that you
cannot use event actions to set the Not Started state. For more information, see Start/Stop Actions option
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
133
Table 72. Understanding sequence states
Sequence
Entry State
Description
•
Start Sequence
•
Active
Not Started
This sequence analyzer has not
started a sequence in the
current user session.
•
Start and Stop Sequence
•
Stopped
•
Start Sequence and Stop Session
•
Stopped
A sequence is in progress.
•
Stop Sequence
•
Stopped
•
Stop Session
•
Stopped
A sequence is stopped, either
by a global event or a sequence
event.
•
Stop Session
•
Stopped
Active
Stopped
Event’s Stop/Start Actions
New Sequence
State
Defining event conditions
Event conditions enable you to tie event actions to things like hit analyzers being triggered and the value of
sequence-level and session-level metrics and custom fields.
IMPORTANT: Event conditions are evaluated only when the sequence is in the Sequence Entry State
specified for the event.
Figure 92. Defining event conditions
You can add, edit, and group event conditions in the same way as other conditions. For more information, see
Configuring conditions.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
134
Defining event conditions
When adding or editing event conditions, you can set the following options.
Table 73. Defining event conditions
Option
Setting the Option
Select the condition type. See Type option for sequence events.
Type
NOTE: The other available fields change depending on the selected Type.
<Type Detail>
The title of this field changes, depending on the option selected in the Type field.
For example, you may be able to select a hit analyzer, a metric, or a custom field.
With Status
Optional—if available, you can specify an analyzer’s status, such as Error. In this
case, the event condition matches only when a hit satisfies both the condition type
and the analyzer status.
not
If available, select to cause an operator to perform its opposite action. For
example, you can change “equal to” to “not equal to” by selecting this check box.
Operator
If available, select an operator that performs the type of match you want to make.
Only the operators that make sense for a selected Type are available. For more
information, see Operators.
Value
If available, specify a numeric or text value (as appropriate).
Extraction Expression
Optional—if available, you can create an extraction expression. For more
information, see “Defining extraction expressions” in the online help.
Type option for sequence events
You can select any of the following values for the Type option.
NOTE: When reviewing the event condition types, keep in mind the following:
•
Some types are listed as “Sequence/Session.” In the software, there are two separate condition
types, one for each.
•
For the “Hit Analyzer” conditions, you can optionally specify an analyzer status. To match the
condition, a hit must trigger the hit analyzer AND have the specified analyzer status.
Table 74. Type option for sequence events
Event Condition Type
Description
Always True
Any hit matches this event condition. When a sequence analyzer has an entry state
of Not Started, you can use this event condition type to trigger an action that starts
a sequence when the first hit is registered.
Hit Analyzer Match
This is the most commonly used condition type. Hits that trigger the specified hit
analyzer also trigger the sequence event.
Hit Analyzer Ordering
You specify two hit analyzers and an operator. The operator defines a relationship
between the two hit analyzers based on the timestamps of the hits that trigger the
hit analyzers. For example, you can create the following set of conditions to
compare the hit analyzers Foo and Bar within a given sequence: if Bar has triggered
AND if Foo has triggered AND if the Bar timestamp is less than the Foo timestamp,
then the event actions are triggered.
IMPORTANT: If a hit analyzer matches more than one hit in a sequence, the hit
analyzer timestamp reflects the most recent match. This may lead to confusing
results if you were not expecting multiple matches.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
135
Table 74. Type option for sequence events
Event Condition Type
Description
Global stop conditions happen under the following circumstances:
Post Sequence: Stopped
by Global Stop
Condition
•
A global stop-sequence event occurred
•
A global stop-session event occurred
•
The maximum session hit count is reached
•
The maximum session duration is reached
•
The “long session” timeout is exceeded
•
The “short session” timeout is exceeded
You can use this condition type to annotate the stopped sequence or perform other
actions.
In most cases, the event that stopped the sequence can also handle other postPost Sequence: Stopped sequence event actions. However, if more than one sequence event can potentially
by Sequence Stop
stop the sequence, you can create an event that uses this condition type and then
Condition
define post-sequence actions in one place, rather than in each individual sequence
event.
You specify a sequence or session custom field, an operator, and a value. The
condition matches when the value of the custom field is updated to the specified
value.
Sequence/Session
Custom Field
•
For session custom fields, the value may be updated by a hit analyzer or this
sequence analyzer.
•
For sequence custom fields, the value of the custom field needs to be
updated by the sequence analyzer to which the event belongs. This allows
you to reference a sequence custom field set by one event in the match
conditions of a later event.
Sequence/Session
Duration
You specify a metric, an operator, and a duration in seconds. The condition matches
when the metric’s value matches the specified duration. Duration is calculated
from the start of the sequence or from the first hit in the session.
Sequence/Session
Hit Counts
You specify the type of hits that should be counted, an operator, and the hit count.
Hit count is calculated from the start of the sequence or session. The condition
matches when the hit count for the sequence or session matches the specified hit
count.
Sequence/Session
Hit Counts by Hit
Analyzer
You specify a hit analyzer and a hit count for a given sequence or session. The
condition matches when the hit analyzer is triggered the specified number of times.
You can use this type of event to identify recurring behavior. For example, you can
create a condition where hits that trigger the hit analyzer Failed Login Page are
counted. When the count reaches the specified hit count, let’s say five failed login
attempts, the event’s action run.
Recording the order of triggered events
You need to assign labels to your events so that Foglight can succinctly record the order in which events were
triggered during a sequence. For more information, see Defining sequence events.
For example, consider a sequence analyzer that has a start event, three intermediate steps, and a stop event,
and uses the labels S, 1, 2, 3, and F, respectively for these events. When viewing a list of sequences, the labels
are used to display the order in which the events occurred, such as Sequence 1 = S,1,2,F (start, event1, event2,
stop) or Sequence 2 = S,3,2,1,F (start, event3, event2, event1, stop).
If an event is triggered more than once (requires that the event’s Allow Multiples check box is selected), the
event label will appear more than once in the sequence record. Using the preceding example, if all the
intermediate events allowed multiple executions, you could conceivably see a session record like this:
S,1,1,2,2,3,3,1,2,3,3,2,1,F.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
136
Defining sequence events
When adding or editing an event, you can set the following options.
Table 75. Defining sequence events
Option
Setting the Option
Name
Specify a unique name.
Label
Specify a label for the event. Labels can be one or two characters. Labels are used
to record the order in which events are triggered within a sequence. For more
information, see Recording the order of triggered events.
Description
Optional—summarize the reason you created the event, including the actions it
performs.
Sequence Entry State
Select the sequence state (Not Started, Active, Stopped) that needs to exist before
the event conditions are evaluated. For more information, see Understanding
sequence states.
Allow Multiples
Select to permit an event to trigger more than once during a sequence. Clear
(default) to restrict the event so that its actions can be run only once in a
sequence. In the latter case, a flag is set to prevent the event from being evaluated
again during the current sequence.
Optional
When available, select to specify that the event does not need to be triggered in
the sequence. Clear (default) to require the event to be triggered before the
sequence ends.
NOTE: Only available when the Events are not required to be sequential check box
is cleared on the Sequence Analyzer - Edit page AND the event’s Sequence Entry
State=Active.
Event Conditions
Add conditions that need to be met before the event’s actions run. For more
information, see Defining event conditions.
Event Actions
By default, the sequence status and the sequence state are not changed when the
event actions run. To change this behavior, see Event actions options.
Sequence Metrics
Optional—for the sequence, add custom metrics and change how topology objects
are created and named. All metrics created for the sequence inherit the pivots
specified for the sequence. For more information, see Working with metrics.
NOTE: Unlike hit analyzers, sequence analyzers always create standard metrics for
the sequence; there is no option to avoid generating them.
Optional—for the event, add custom metrics and change how topology objects are
created and named. All metrics created for the event inherit the pivots specified
Sequence Event Metrics for the sequence. For more information, see Working with metrics.
NOTE: Unlike hit analyzers, sequence analyzers always create standard metrics for
sequence events; there is no option to avoid generating them.
Custom Fields
Optional—update the value of custom fields. For more information, see Updating
custom fields.
Scripts
Optional—execute a script. For more information, see Executing scripts.
Event actions options
By default, the sequence status and the sequence state are not changed when the event actions run.
Set Sequence Status option
By default, all sequences have a default status of OK. If you create a sequence analyzer to detect problems
within a series of steps, you can use the Set Sequence Status event action to change the sequence status to
Warning or Error.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
137
Start/Stop Actions option
By default, an event does not change the state of the sequence. For more information, see Understanding
sequence states.
NOTE: Before choosing an event action that stops a session, you need to be sure that the web application
ends the session under the conditions defined in the event conditions. For example, you can stop a session
when an end user clicks a logout link, rather than waiting for the session to eventually time out.
You can select any of the following values for the Start/Stop Actions option.
Table 76. Start/Stop Actions option
Start/Stop Actions
Description
New Sequence State
No Change (default)
Leaves the sequence state in its current state.
Unchanged
Starts a new sequence. By default, the first event
you create for a sequence has the Start Sequence
action assigned to it.
Active
Start Sequence
Stop Sequence
Stops the current sequence.
Stopped
Start and Stop Sequence
Starts a sequence and immediately stops it. This can Stopped
be used to identify activities such as repeated failed
logins.
Stop Session
Stops the sequence and the session.
Stopped
Start Sequence and
Stop Session
Starts a sequence, immediately stops it, and then
ends the session.
Stopped
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring analyzers
138
5
Capturing traffic outside the
network
Sniffers and Archivers are exceptionally good at monitoring and capturing web traffic occurring within a
physical or virtual network. And while they can capture data about calls out to external resources and back to
internal web servers, they have little insight into the activities occurring on those external resources. For
example, a Sniffer cannot measure how long it takes for a content delivery network (CDN) server to respond to
a request for content. The Sniffer can track only the time spent by an in-house web server responding to the
initial page request (Initial Response Time), before the request is passed on to a CDN server to provide the page
content—which obviously does not reflect an end user’s experience of how long it took for the page to load.
Browser instrumentation is a small JavaScript library, packaged as a single file, and included in a web page for
the purposes of monitoring page performance and application health from inside the browser. This section helps
you decide when to implement browser instrumentation, and describes how to set up an instrumented
monitoring solution:
•
Deciding when to add browser instrumentation
•
Introducing the instrumentation components
•
Configuring an instrumented monitoring solution
•
Verifying instrumentation payloads are received
•
Viewing metrics collected by instrumentation
Deciding when to add browser
instrumentation
You may want to add browser instrumentation when monitoring the following types of web sites and web
applications.
•
Web 2.0/AJAX Application — In Web 2.0 applications, the client’s browser does most of the processing
for the application and also maintains most of the information about the state of the client. For
example, an AJAX-based application makes asynchronous XMLHttpRequest calls to the web server, and
the web server responds with fragments of content or raw data, which are then interpreted by the
client-side JavaScript. To monitor the performance and health of the client’s browser, you need to
instrument web pages in the application.
•
Mashup Web Site or Application — A mashup web site or web application combines first-party dynamic
content with dynamic content from third parties, such as social media sites (for example, Twitter or
Facebook) and remote advertising servers. With all this extra content outside the control of your site or
application, you cannot fully understand your end users’ experience of these pages without the extra
information provided by browser instrumentation.
•
Cloud-based Web Site or Application — Cloud environments are complex, dynamic, and elastic, which
makes Sniffer deployments impractical. Browser instrumentation is currently the only way to monitor
cloud-based web sites or web applications.
•
Content Delivery Network (CDN) — When monitored web sites or web applications use a CDN to deliver
static content, a Sniffer cannot measure how long the CDN servers take to respond to a request for an
Foglight APM 5.9.9 Administration and Configuration Guide
Capturing traffic outside the network
139
individual piece of content. Browser instrumentation provides the missing timing data, which in turn
enables a Sniffer to report more accurate values for page load metrics.
Introducing the instrumentation
components
The Foglight approach to instrumentation is similar to other types of instrumentation you may have used for
analytics. Foglight instrumentation has two components: the instrumentation library and the instrumentation
listener. To learn how to implement instrumentation, see Configuring an instrumented monitoring solution.
Instrumentation library
The instrumentation library is a JavaScript file approximately 10 KB in size. The library contains a set of APIs to
collect browser event information and capture content displayed in the browser. It also implements the W3C
Navigation Timing interface, described in http://www.w3.org/TR/2012/REC-navigation-timing-20121217/, to
collect performance data for browsers that support this interface. For a list of the details captured by the
instrumentation library, including the Navigation Timing properties used to calculate them, see
“Instrumentation and navigation timing metrics” in the Foglight APM Reference Guide.
TIP: Most current browser versions support the Navigation Timing API. To find out which earlier versions
support the API, search the Internet for “navigation timing api browser support.”
The instrumentation library loads as part of the initial page load. As the page executes, the script tracks
JavaScript errors and logging data as well as the navigation timing data. The scripts assigns the header X1EUQuestInstrumentation to instrumentation hits, batches the data into instrumentation payloads, and
forwards payloads asynchronously to the instrumentation listener. If the instrumented page uses SSL, the
payload is sent to the listener using SSL.
The instrumentation library does not interfere with normal page execution, and when using the default script
settings, script activity has a negligible affect on page performance.
Instrumentation Listener
The instrumentation listener is the host URL of the original HTML page. If a Sniffer is already monitoring the
web site or application you want to instrument, you do not need to do anything else to set up an
instrumentation listener. When the instrumentation library sends an instrumentation payload to the web server
using the POST command, the server sees the post and returns a response. The Sniffer monitors this
conversation, and sends the captured data to Archivers for further processing and storage as instrumented
details and metrics for that page hit.
Configuring an instrumented monitoring
solution
To track client-side events and performance data, you need to instrument monitored web sites or applications
and also ensure that a Sniffer is monitoring the same site or application. You can instrument an entire web site
or application (by including the instrumentation library in a global header), or you can instrument only the
pages you are interested in.
Foglight APM 5.9.9 Administration and Configuration Guide
Capturing traffic outside the network
140
By default, the instrumentation library posts its payloads to <host_URL>/empty.txt. For example, if you
instrument the web site www.dell.com, the library posts to www.dell.com/empty.txt.
IMPORTANT: You must deploy an empty file named empty.txt onto your web site so that, when the
instrumentation posts data to the web site, it will not result in the Object Not Found (404) response code.
Failing to do so results in lots of 404 errors being generated after you deploy the javascript
instrumentation.
To configure an instrumented monitoring solution:
1
In Foglight, download the instrumentation library.
a
In the navigation panel, under Dashboards, click Administration > Cartridges > Components for
Download.
b
Click JS Instrumentation.
c
Click Download.
d
Follow your browser’s prompts to save the instr.js file.
2
Copy instr.js to the same location as other scripts used by your monitored web site or application.
3
In the text editor of your choice, open a file where you want to add a call to the script.
Do one of the following:
4
•
To monitor an entire site/application, open an appropriate global header file.
•
To monitor individual web pages, open a page file.
Add the <script> statement shown in the following code snippet to the file, substituting in the relative
path to the script in your environment. Enclose the <script> tag within the <head> tag as close to the
top as possible.
IMPORTANT: For the id attribute, ensure that you specify "--foglight-script--" exactly as
shown, including all dashes.
<head>
...
<script
type="text/javascript" src="<path>/instr.js" id="--foglight-script--">
</script>
...
</head>
5
Save the file.
6
If you are monitoring individual web pages, copy the <script> statement to the other pages.
7
If you are not sure that one of your Sniffers is monitoring the instrumented site or application, check the
Web Sites and Endpoints dashboard.
a
Log in to Foglight.
b
On the navigation panel, under Dashboards, click APM > Web Sites and Endpoints.
c
If the Web Sites quick view is not showing, click the Web Sites tile.
d
Verify that the instrumented web site or application is in the list of Web Sites.
Foglight APM 5.9.9 Administration and Configuration Guide
Capturing traffic outside the network
141
Verifying instrumentation payloads are
received
When the instrumentation library sends an instrumentation payload to the web server using the POST command,
the server sees the post and returns a response. The Sniffer monitors this conversation. You can verify that the
conversation is occurring by searching hits.
NOTE: By default, the instrumentation library posts its payloads to <host_URL>/empty.txt.
To verify that instrumentation payloads are received:
1
On the navigation panel, under Dashboards, click APM > Search > Hits.
2
Click Simple Search.
3
From the Content Category list, select Instrumentation.
4
Click Search.
The results table displays instrumentation payload hits. If the table is empty, ensure that the web site is
in use and wait a few minutes before re-running the search. If the table is still empty, review your
instrumentation setup for issues.
Viewing metrics collected by
instrumentation
You can view hits that contain instrumentation data from the Search Hits dashboard. Instrumentation data
includes navigation timing metrics—client-side metrics collected from the browser. Navigation timing metrics
include things like content delivery time and page completion times. For metric definitions and explanations of
calculated data, see “Navigation timing metrics” in the Foglight APM Reference Guide.
To view hits with instrumentation data:
1
On the navigation panel, under Dashboards, click APM > Search > Hits.
2
Click Simple Search.
3
From the Content Category list, select HTML.
4
From the Hit Analyzer list, select a hit analyzer that you know is collecting metrics for instrumented web
pages.
5
Click Search.
Hits on instrumented pages appear in the results table. Values for the following calculated navigation
timing metrics are displayed for the HTML pages that include the instrumentation JavaScript.
Foglight APM 5.9.9 Administration and Configuration Guide
Capturing traffic outside the network
142
•
Navigation Page Completion Time—the total time taken from the beginning of a navigation
action (such as a click on a hyperlink) to the end of the load event when the page has been
completely displayed in the browser. For an example, see “Tracking time spent on a page using
instrumentation” in the Foglight APM Reference Guide.
•
Navigation Content Delivery Time—the amount of time from the beginning of the request to the
beginning of the response. For an example, see “Tracking time spent on a hit using
instrumentation” in the Foglight APM Reference Guide
6
To view all navigation timing details for an instrumented hit, in the Hits column, click Hit Detail View
and open the page in a new window.
7
Scroll to the bottom of the page to see the Navigation Timing Details view.
Foglight APM 5.9.9 Administration and Configuration Guide
Capturing traffic outside the network
143
6
Configuring advanced options
The advanced options affect the software components and user interface elements.
Advanced configuration options include the following activities:
•
Configuring internal settings
•
Substituting static page elements
•
Excluding subnets from data collection
Configuring internal settings
The Settings dashboard contains configuration options that affect how Foglight APM components operate. Avoid
modifying these options until you understand how the changes affect your installation.
TIP: For settings that affect how real user data is captured, transformed, analyzed, and stored, see
Configuring traffic analysis.
The dashboard organizes the settings into three groups based on which components are affected: Archiver query
settings, Replay settings, and Metadata settings.
Archiver query settings
To modify settings, navigate to APM > Configure Advanced Options > Settings. The first few Archiver Query
settings affect interactions between Archivers and the management server. The remaining settings affect the
time taken for searches on the distributed Archiver database.
The following table describes all the options. Remember to save after making changes.
Table 77. Archiver query settings
Option
Setting the Option
Network Keep Alive
Sets how long server network connections stay connected to Archivers with no
active communication. Valid values are 10,000 to 100,000,000 milliseconds.
Hit Search Result Limit
Sets the maximum number of hits that can be returned from a single Archiver in
response to a single hit search. Default is 200 per Archiver searched. For example,
in an installation with three Archivers and a result limit of 200 hits per search, up to
600 hits may be returned in one search. Value must be an integer. Larger values
may increase the time required to complete a search.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring advanced options
144
Table 77. Archiver query settings
Option
Setting the Option
Session Search Result
Limit
Sets the maximum number of sessions that can be returned from a single Archiver in
response to a single session search. Default is 200 per Archiver searched. For
example, in an installation with three Archivers and a result limit of 200 sessions
per search, up to 600 sessions may be returned in one search. Larger values may
increase the time required to complete a search.
Sets the maximum number of sequences that can be returned from a single Archiver
in response to a single sequence search. Default is 200 per Archiver searched. For
Sequence Search Result
example, in an installation with three Archivers and a result limit of 200 sequences
Limit
per search, up to 600 sequences may be returned in one search. Larger values may
increase the time required to complete a search.
Search Time Range
Limit
Sets the maximum time range that can be selected for a search. All Search dialog
boxes inherit this setting. For example, if you select last hour for this setting,
anyone who runs a search can select only time ranges from last minute to last hour
in the Search dialog boxes. Larger values may increase the time required to
complete a search.
Replay settings
To modify settings, navigate to APM > Configure Advanced Options > Settings. Replay settings affect how user
sessions are replayed.
Figure 93. Replay settings
The following table describes all the options. Remember to save after making changes.
Table 78. Replay settings options
Option
Setting the Option
Page Element Path Endings
Specifies a comma-separated list of URL path endings (no spaces) that should
always be treated as elements of a page. A URL path ending can be a file
extension or any text that ends a URL. Any query parameters appended to path
endings are ignored.
Example: .gif,.jpg,.png,.ico,.pdf,.swf,.js,.css,
.xml,.xsl,/dynamicallyGeneratedImage
Rewrite JavaScript
Select Yes to enable JavaScript when replaying captured HTML hits. Because
JavaScript may sometimes cause problems during replay, the default is No.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring advanced options
145
Table 78. Replay settings options
Option
Setting the Option
Disables JavaScript functions that may interfere with replay. To list a function
Disable JavaScript Functions that should not run during replay, click Add and specify the name of the
JavaScript function.
Treat JavaScript Variable
Names as Anonymous
Functions
Disable Google Analytics
During Replay
Select (default) to detect all variables which are instances of functions and
disable them. Clear to enable these functions.
Select (default) to disable callbacks from Google® Analytics to avoid sending
duplicate data to Google Analytics. Clear to enable Google Analytics during
replay.
Less Strict Replay with User Select Yes (default) to improve the replay of user input data when JavaScript is
Input (Forms)
used to manipulate form field values. Select No if JavaScript is never used.
Less Strict Replay with User Select Yes to attempt to improve the replay of user clicked links when hit
Input (Links)
referers are not present. Default is No.
Display Mode
Specifies the content to display when first opening the Replay browser. Default
is Response Replay. For a description of each of the available values, see Values
for Display Mode (following this table).
User Input Highlighting Primary Color
Sets the primary color to be used when highlighting user input. You can specify
any valid CSS color value, such as #FC0, RGB(255,204,0), or lime.
User Input Highlight Secondary
Sets the secondary color to be used when highlighting user input. This color is
used for highlighting specific items, such as the option a user selected from a
pull-down menu. You can specify any valid CSS color value, such as #FF0,
RGB(255,255,0), or yellow.
Select Yes to use page elements, such as images and stylesheets, from
unrelated sessions when replaying sessions. Default is No.
Use Elements from
Unrelated Sessions
NOTE: Enabling this option may cause unexpected results if more than one
element shares the same file name. You may prefer to have the Replay browser
substitute locally-stored static page elements. For more information, see
Substituting static page elements.
Specifies the action to take when page elements, such as images, JavaScript,
and CSS files, are not available for the session being replayed and cannot be
found in static page elements (if defined) or in other sessions (when the Use
Elements from Unrelated Sessions option is enabled).
The following actions are available:
When Element Content not
Available
•
Redirect to ‘live’ web server (default)—retrieves page elements from
the live web server. Select this option only when the live server is
reachable from the Replay browser and the server does not require a
login to reach the page elements. This option may cause unexpected
results if more than one element in the live traffic share the same file
name.
•
Send 404 Response—sends a 404 error code to the Replay browser. For
missing images, a broken image icon is displayed. For missing style
sheets or JavaScript, the HTML page may not render correctly.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring advanced options
146
Values for Display Mode
Table 79. Values for Display Mode
Display Mode
Description
Response Replay
Displays an approximation of the HTTP hit’s response content as the end user
originally viewed it. If the response content is not available for any of the page
elements (for example, if the browser used a cached image), the Replay browser
refers to the Use Elements from Unrelated Sessions and When Element Content not
Available options for next steps.
Response Replay (Show
Submitted Input)
Displays hits similarly to Response Replay mode, except that the final values for any
form fields where user entered data are populated with the data that was
submitted to the web server. Any links or submit buttons clicked to move to the
next page are highlighted as well. Submitted user input cannot be shown for forms
or links that are manipulated by JavaScript on the client browser, including actions
that are purely client-side.
Response Source
Displays the source code that was captured for HTML hits. Useful for debugging
dynamic web applications. If a selected hit is not an HTML page, the result is
equivalent to that displayed using the Response Replay mode.
Hit Details
Displays network and HTTP details about the selected hit, including response time,
hit analysis, request and response fields, and headers, cookies, and Archiver
details.
Request Content Body
Displays as text the HTTP request message body of the hits that use POST as the
HTTP request method. Useful for viewing data POST-ed from the browser, including
special request content-types (for example, XML and JSON formats).
Instrumentation
Displays raw instrumentation data. Requires that an APM Administrator
implemented the Foglight instrumentation JavaScript library on the monitored web
site or application.
Metadata settings
To modify settings, navigate to APM > Configure Advanced Options > Settings. Metadata settings affect the
content of choosers and Captured Metadata lists.
Figure 94. Metadata settings
The following table describes all the options. Remember to save after making changes.
Table 80. Metadata settings options
Option
Setting the Option
Metadata Cache
Timespan
Sets the amount of time between updates to the server’s metadata cache. Valid
values are 1,000 to 1,000,000 milliseconds.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring advanced options
147
Table 80. Metadata settings options
Option
Setting the Option
Metadata Sensitivity
Sets the number of times the name of a particular request field, request header,
response header, or cookie must appear in a given database segment before it is
considered significant enough to be returned by metadata queries. Valid values are
1 to 1,000,000 hits per segment, where 1 returns all values, regardless of how often
they occur.
NOTE: If you use an extremely large number of unique names (for example,
dynamically-generated field names), you should set this field to a higher number,
such as 5000, to avoid excessively long metadata lists.
Specifies file extensions to ignore when generating the list of paths captured by the
system for display in the Path chooser and Captured Path metadata list. This field
Filtered Path Extensions must be a comma-separated list of extensions (no spaces, no dots).
Example: gif,jpg,png,ico,bmp
Substituting static page elements
Static page elements are resources, such as images and JavaScript, stored on a local drive (not in the Archiver
database) and substituted during session replays when the original resource is not available in the session data.
You can use static page elements to provide a richer, more accurate session replay experience. For more
information, see “Replaying hits, sequences, and sessions” in the Foglight APM User Guide.
For example, when a browser caches a resource, such as the image of a company logo on the initial visit to a
web site, the web server no longer needs to serve that image on subsequent hits in other sessions. If you
attempt to replay a user session that references a cached image, by default the replayer checks the live web
traffic to see if it can find the missing image. If it does, it displays the image. If not, it displays a broken image
icon. In either case, the session replay may not be identical to the session experienced by the user. By defining
a static page element, the session replay can use a local copy of the original image, which makes for a more
authentic replay experience.
You can also use static page elements to substitute resources that are not served by the web servers being
monitored. For example, you may have some page elements being served by an external content delivery
network (CDN). Because hits in a CDN cannot be captured by Sniffers, they cannot be replayed without
accessing the original content source. You can provide the original content as a static page element.
TIP: You have other alternatives for handling missing content. Instead of looking for resources in live
traffic (resources that may have changed since the session was recorded), you can choose to always return
a 404 error. Or you can have the session replay look for missing resources in unrelated sessions, which may
identify the session during which the original image was retrieved by the web server and cached by the
browser. For more information, see the When Element Content not Available and Use Elements from
Unrelated Sessions options in Configuring internal settings.
To configure Static Page Elements, navigate to APM > Configure Advanced Options > Static Page Elements.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring advanced options
148
Figure 95. Static Page Elements dashboard
In the Static Page Elements dashboard, you can add, edit, and remove static page elements using the standard
controls. You can also view the contents of an uploaded resource file.
For more information, see the following topic:
•
Defining static page elements
Defining static page elements
When adding or editing a static page element, you can set the following options.
Table 81. Static Page Elements options
Option
Setting the Option
Upload File
Specify the resource file, such as an image, JavaScript, or a CSS file, to upload from
your local server. When editing a static page element definition, you can either use
the existing resource or select a new resource.
Name
Use the default name (which is the file name of the resource you uploaded) or
specify a new name. The name is for display purposes only—the uploaded resource
remains the same.
Description
Optional—summarize the purpose for this resource. For example, if the resource is
unavailable because pages are being served by a content delivery network (CDN),
you may want to note that here.
Content Type
Use the default provided when you upload the resource, unless you know that the
content type is incorrect. This value is included in the HTTP response when this
page element is displayed during replay.
URL
Use the default Ends With: <resource file name> or define a new URL rule using any
of the available operators and a full or partial URL. When URLs matching this rule
are found as subelements of a page, the replay uses this resource instead of looking
elsewhere for the page element.
Start Time
Optional—specify the time when this resource went live on the production web
server. All hits with a timestamp greater than or equal to this time (but less than or
equal to the End Time, if specified) use this resource during replays.
End Time
Optional—specify the time when this resource was removed or replaced on the
production web server. All hits with a timestamp less than or equal to this time (but
greater than or equal to the Start Time, if specified) use this resource during
replays.
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring advanced options
149
Example
Figure 96. Editing static page elements
Excluding subnets from data collection
If you are not interested in capturing data for a subnet in your network, you can exclude the subnet by defining
a Subnet Filter. Sniffers ignore all traffic associated with the subnet, which means the collected metrics no
longer reflect hits and sessions occurring within the subnet.
To configure Subnet Filters, navigate to APM > Configure Advanced Options > Subnet Filters.
Figure 97. Subnet Filters dashboard
In the Subnet Filters dashboard, you can add, edit, and remove subnet filters using the standard controls.
For more information, see the following topic:
•
Defining subnet filters
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring advanced options
150
Defining subnet filters
When adding or editing a subnet filter, you can set the following options.
Table 82. Subnet Filters options
Option
Setting the Option
Name
Specify a unique name for the subnet.
Description
Optional—summarize the reasons for removing this subnet from the traffic
capture.
CIDR Block
Specify an IP address and subnet mask, separated by a forward slash
(/). For example: 177.18.1.0/7
Example
Figure 98. Adding subnet filters example
Foglight APM 5.9.9 Administration and Configuration Guide
Configuring advanced options
151
7
Integrating Foglight APM with other
products
You can integrate Foglight APM with CA SiteMinder Policy Servers (authorization and authentication services)
and SafeNet Hardware Security Modules (cryptographic keys). The integration steps are described in the
following sections.
NOTE: You can also integrate with Dell NetVault Backup (backup and recovery). For instructions, see the
Foglight APM Installation and Setup Guide.
Available integrations:
•
Integrating with CA SiteMinder
•
Integrating with SafeNet Hardware Security Modules
Integrating with CA SiteMinder
CA SiteMinder is a centralized web access management system that enables:
•
User authentication and single sign-on
•
Policy-based authorization
•
Identity federation
•
Auditing of access to web applications and portals
The SiteMinder Policy Server is a SiteMinder component that provides authorization and authentication services
to applications. To monitor applications that use SiteMinder, you need to configure Foglight APM to perform the
same type of user authentication with SiteMinder that your applications currently perform with SiteMinder.
To manage SiteMinder servers, navigate to APM > Configure Advanced Options > SiteMinder Servers.
Figure 99. SiteMinder Servers dashboard
Foglight APM 5.9.9 Administration and Configuration Guide
Integrating Foglight APM with other products
152
In the SiteMinder Servers dashboard, you can add, edit, and remove server definitions using the standard
controls. You can also test the connection to a SiteMinder server.
In the SiteMinder Servers dashboard, you can perform the following tasks:
•
Adding SiteMinder Server definitions in Foglight
•
Registering Foglight as a SiteMinder agent
•
Adding SiteMinder to a sessionizing policy
Adding SiteMinder Server definitions in Foglight
Create a definition for each SiteMinder Policy Server that controls access to the applications you want to
monitor.
NOTE: To ensure that you capture accurate session details, remember to add the SiteMinder cookie to
your sessionizing policy. For more information, see Adding SiteMinder to a sessionizing policy.
To add a SiteMinder server definition:
1
On the navigation panel, under Dashboards, click APM > Configure Advanced Options > SiteMinder
Servers.
2
Click Add.
3
In the Add SiteMinder Server dialog box, configure the following options.
Table 83. SiteMinder Server options
Agent Name
Specify a name, such as foglight_agent. You refer to this name when configuring the
SiteMinder Policy Server. See Registering Foglight as a SiteMinder agent.
Description
Optional—specify additional information, such as where the SiteMinder Policy Server
is located or the list the applications that use this SiteMinder server.
IP Address
Specify the IP address of the host on which the SiteMinder Policy Server is installed.
Accounting Port
If the default accounting port was changed during the SiteMinder installation, check
the port setting in the Policy Server Management Console and enter that port
number in this box. Default is 44441.
Authentication Port
If the default authentication port was changed during the SiteMinder installation,
check the port setting in the Policy Server Management Console and enter that port
number in this box. Default is 44442.
Authorization Port
If the default authorization port was changed during the SiteMinder installation,
check the port setting in the Policy Server Management Console and enter that port
number in this box. Default is 44443.
Shared Secret
Specify the key used for initial connection to the SiteMinder Policy Server. The
SiteMinder Policy Server can be configured to assign a new dynamic shared secret to
Foglight.
Associated Sniffers
Select the Sniffers that require access to this SiteMinder server. Recall that in a
multiple-appliance installation, Sniffers may be installed in different locations
(geographically or on the network) and therefore require access to different
SiteMinder Policy Servers and applications. For more information, see Managing
Sniffers.
Foglight APM 5.9.9 Administration and Configuration Guide
Integrating Foglight APM with other products
153
Figure 100. Add SiteMinder Server dialog box
4
Click Save.
5
In the SiteMinder Servers dashboard, select the check box for the newly created server definition and
click Test.
If the test fails, click Edit
and correct the SiteMinder Server definition.
Registering Foglight as a SiteMinder agent
After the SiteMinder server is defined in Foglight, a SiteMinder Administrator can create an agent definition for
Foglight. In the SiteMinder Agent Dialog page, specify the following values.
Table 84. SiteMinder Agent options
SiteMinder Agent Option
Setting the Option
Name
Specify the Agent Name you specified when defining the SiteMinder Server
definition. See Adding SiteMinder Server definitions in Foglight.
Support 4.x agents
Select this check box.
IP Address or Host Name
Specify the IP address of the Foglight Management Server.
Secret
Specify the Shared Secret you specified when defining the SiteMinder Server
definition. See Adding SiteMinder Server definitions in Foglight.
Adding SiteMinder to a sessionizing policy
Applications that use SiteMinder identify user sessions through the default SMSESSION or GMWSESSION cookies
or through some other cookie defined within SiteMinder. To capture accurate session details, you need to add
the cookie used by your SiteMinder servers to the default sessionizing policy.
•
To add the cookie, you need to define a session identifier with the name of the cookie and set the Type
to SiteMinder. For more information, see Sessionizing policies—Identifying user sessions.
•
To capture user names associated with SiteMinder sessions, you need to define a username rule with the
same cookie you added as a session identifier. For more information, see Sessionizing policies—
Associating user names with user sessions
Foglight APM 5.9.9 Administration and Configuration Guide
Integrating Foglight APM with other products
154
Integrating with SafeNet Hardware
Security Modules
SafeNet Hardware Security Modules (HSMs) store and manage cryptographic keys, providing organizations with
secure encryption, decryption, authentication, and digital signing services. When integrated with a SafeNet
HSM, Foglight can use the keys stored within the HSM server to decrypt HTTPS traffic. Foglight uses the keys in
a secure manner consistent with the SafeNet HSM model.
To integrate Foglight with SafeNet HSM servers, navigate to APM > Configure Advanced Options > HSM
Devices.
Figure 101. HSM Devices dashboard
In the HSM Devices dashboard, you can add, edit, and remove HSM servers using the standard controls. You can
also generate the client certificate that needs to be uploaded to the HSM server.
In the HSM Devices dashboard, you can perform the following tasks:
•
Planning a Foglight–HSM integration
•
Registering HSM servers in Foglight
•
Generating SafeNet HSM client certificates
•
Associating HSM keys with monitored IP addresses
Planning a Foglight–HSM integration
For each Foglight Sniffer, you need the IP address of the appliance hosting the Sniffer.
For each HSM appliance, you need the following information:
•
HSM server name
•
HSM server IP address
•
HSM server certificate location — You can request the HSM server certificate from your HSM
Administrator. If you are managing your own HSM appliance, refer to the SafeNet documentation for
instructions on how to retrieve the server certificate.
•
A list of the SSL ports on your network with the HSM partition, password, and private key used
When you have this information, you are ready to begin defining HSM servers.
Foglight APM 5.9.9 Administration and Configuration Guide
Integrating Foglight APM with other products
155
Registering HSM servers in Foglight
To add an HSM server definition:
1
On the navigation panel, under Dashboards, click APM > Configure Advanced Options > HSM Devices.
2
Click Add.
3
In the Add HSM Device dialog box, configure the following options:
Table 85. HSM Device options
Name
Specify the same name as the HSM server.
Description
Optional—specify additional information, such as where the HSM server is located or
the list the applications that use the HSM server.
IP Address
Specify the IP address of the HSM server.
HSM Certificate
Browse to and select the HSM server certificate that you received from your HSM
Administrator.
Associated Sniffers
Select the Sniffers that require access to this HSM server. Recall that in a multipleappliance installation, Sniffers may be installed in different locations
(geographically or on the network) and therefore require access to different HSM
servers. For more information, see Managing Sniffers.
Figure 102. Add HSM Device dialog box
4
Click Save.
Generating SafeNet HSM client certificates
The HSM appliance requires client certificates for all applications that need to interact with an HSM server. You
need to generate an HSM client certificate for each Sniffer appliance that needs access to an HSM server. The
client certificate is created using the IP address of the Sniffer appliance.
NOTE: If you ever change the IP address of a Sniffer appliance that has been integrated with an HSM
server, you will need to regenerate the HSM client certificate.
To generate a certificate:
1
In the HSM Devices dashboard, click Certificates.
The HSM Client Certificate Wizard opens. Use the Next and Previous buttons to navigate between
screens.
2
In the Select Sniffer screen, specify the Sniffer.
Foglight APM 5.9.9 Administration and Configuration Guide
Integrating Foglight APM with other products
156
3
In the Confirm IP Address screen, verify that the displayed IP address matches the true IP address of the
appliance hosting the selected Sniffer, rather than an alias.
For example, if the IP Address is 127.0.0.1, you need to replace it with the actual IP address of appliance
hosting the Sniffer.
4
In the Create Certificate screen, click the link. Navigate to a directory, and save the client certificate
file.
5
Click Finish.
6
After you generate client certificates for all the Sniffers that require access to HSM servers, send the
certificates to your HSM Administrator. Your HSM Administrator uploads the client certificates to the HSM
appliance, registers them using the IP address of the appliances (not the hostnames), and grants access
to the partitions containing the required private keys.
Associating HSM keys with monitored IP
addresses
First you need to define the HSM keys you want to use in the Private Keys dashboard. For more information, see
Managing HSM private keys.
Then, for each monitored IP address requiring an HSM key, associate each of the server’s SSL ports with the
correct HSM key. You need the password for the HSM partition. For more information, see Monitoring secure
(HTTPS) web traffic.
Foglight APM 5.9.9 Administration and Configuration Guide
Integrating Foglight APM with other products
157
8
Maintaining and troubleshooting
You should monitor your appliances to ensure that they are performing optimally. If you discover any issues, you
can use the features in the Support menu to help you resolve them. This section explains how to check the
health of the appliances, roll back to previous configurations, generate support bundles for Dell Support, and
update the appliance software.
NOTE: The Installation and Setup Guide also contains maintenance and troubleshooting tasks that can be
completed from the Console Program or the command line, including a Sniffer Diagnostic utility that can
validate the performance of the monitoring NICs.
Maintenance and troubleshooting include the following activities:
•
Checking the health of Foglight APM Appliances
•
Managing configuration changes
•
Creating support bundles
•
Updating appliance software
•
Restarting the embedded Agent Manager
Checking the health of Foglight APM
Appliances
You can monitor the health of the Foglight APM Appliances in your network and the performance of the Sniffers,
Archivers, and Relayers. To view a summary of installed appliances and the software components they host,
navigate to APM > Support > Appliance Health.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
158
Figure 103. Appliance Health dashboard
The tiles across the top of the dashboard summarize the health of the appliances, Sniffers, Archivers, and
Relayers. For example, let’s look at the Appliances tile. You can see how many appliances you have in the upper
right corner.
Figure 104. Appliance Health tiles
NOTE: If you have a single All-in-One Appliance installed, then your tiles should show one appliance and
one of each of the software components.
The severity indicators across the bottom of the tile show how many appliances are in each of the severity
states. The following table summarizes the severity levels.
Table 86. Appliance health severity levels
Icon
Severity
Description
Normal
The component is operating within normal thresholds. A normal severity level
indicates that there have been no critical, warning, or fatal events fired. Foglight
does not record events that are successful; it can only determine that there are no
events that had problems.
Warning
Represents a possible performance problem, based on calculations on current server
metrics against best-practices thresholds.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
159
Table 86. Appliance health severity levels
Icon
Severity
Description
Critical
Indicates that the current metric values point strongly towards performance-related
problems with the specified component.
Fatal
There is a very strong indication that the server is experiencing conditions that can
degrade performance.
When any of the tiles show an appliance or software components in a non-normal state, you need to investigate.
For more information, see the following topics:
•
Investigating issues with appliance health
•
Summary of Appliance Health views
Investigating issues with appliance health
The investigative process is the same for appliances and the software components. This workflow uses the
Appliances tile as an example. For a list of views that open for the other tiles, see Summary of Appliance Health
views.
NOTE: This workflow describes how to use the dashboard to investigate issues. For reference information
about the purpose of each view and the metrics displayed, see “Appliance Health Dashboard” in the
Foglight APM Reference Guide online help.
To investigate issues:
1
Select a tile that shows an appliance/component in a non-normal state.
For this example, we investigate the Appliance tile, which is the default selection. The Appliances quick
view appears below the tiles, displaying all appliances in your installation.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
160
The Summary - All Appliances view contains a table that summarizes the key performance indicators for
each of the appliances, including percentages for CPU statistics and counts for running and blocked
processes. The Alarms view displays alarms (if any) for all appliances.
2
If an alarm exists in the Alarms view, drill down on the alarm.
3
After alarms are resolved, if you still have an appliance/component in a non-normal status, select it.
For example, if you select an appliance, the view changes to show some statics for the selected
appliance.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
161
From the Appliance: SelectedAppliance view, you can review how the appliance is accessing CPU and
memory, as well as the various file systems. Alarms appear across the bottom.
Let’s assume this view does not provide enough detail to reveal the issue. You can drill down to more
detailed metrics.
4
Click Explore to open a dashboard for the selected appliance/component.
For example, if you explore from the Selected Appliance view, the Appliance: SelectedAppliance
dashboard opens, displaying a Summary tab for the selected appliance.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
162
Review the performance information to look for problems with the appliance. To see even more detail
about the views in the Summary tab, click one of the other tabs: CPU tab, Memory tab, Storage tab,
Processes tab.
5
After you identify the issue, take action to resolve the issue discovered.
For example, if the issue is with an appliance, a resolution may involve contacting Dell Support,
upgrading the appliance, adding new appliances to handle increased load, or repairing/replacing ailing
appliances.
Summary of Appliance Health views
The following table provides links to the dashboards and views you see when investigating issues.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
163
Table 87. Appliance Health views
Selected Tile
Quick View
Clicking Explore opens:
Initially, the quick view displays the Summary - All
Appliances view. When an appliance is selected, it
shows the Appliance: SelectedAppliance view.
Appliance: SelectedAppliance
dashboard
Initially, the quick view displays the Summary - All
Sniffers view.
When a Sniffer is selected, it shows the Sniffer:
SelectedSniffer view.
Sniffer: SelectedSniffer dashboard
Initially, the quick view displays the Summary - All
Archivers view. When an Archiver is selected, it
shows the
Archiver: SelectedArchiver view.
Archiver: SelectedArchiver dashboard
Initially, the quick view displays the Summary - All
Relayers view. When a Relayer is selected, it shows
the
Relayer: SelectedRelayer view.
Relayer: SelectedRelayer dashboard
Managing configuration changes
The Configuration Change Log incrementally records all changes to the traffic capture and traffic analysis
options, including what changed and who made the change. Each changed configuration is assigned a version
number. The Configuration Change Log keeps the most recent 100 versions. You can save a version permanently
by exporting the configuration to a file.
TIP: If you notice a sudden change in the metrics gathered by the appliance, you may be able to trace the
cause using the Configuration Change Log. Review recent versions to see if you can link the change in data
capture to a recorded configuration change.
To view and manage changes to the configuration, navigate to APM > Support > Configuration Change Log.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
164
Figure 105. Configuration Change Log dashboard
In the Configuration Change Log dashboard, you can export and import configuration changes and review
warnings. You can also view more details about a selected version and optionally roll back some or all of the
configuration changes to match the selected version. For procedures, see the online help.
For more details, see the following topics:
•
Reviewing change versions
•
Understanding configuration modules and objects
•
Exporting configuration modules and objects
•
Restoring and importing configurations
•
Reviewing warnings
Reviewing change versions
To see what changed in a particular version, click its Details
icon. The Details dashboard opens, displaying
details that compare the selected version to the current version.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
165
Figure 106. Configuration Change Log details
Information view
The Information view contains identifying information about the selected version, including the version number,
the name of the user who made the change, the type of change, the object that was modified, and the
timestamp of the change.
Objects Defined view
The Objects Defined table describes all the configuration objects in the selected version.
Table 88. Objects Defined view
Objects Defined
Description
Object Type
Shows the type of each object, such as Hit Analyzer or Custom Field.
Object Name
Shows the display name for each of the objects.
In Current
Contains a Boolean value that indicates whether an object still exists in the most
current configuration (true) or not (false).
Same As Current
Contains a Boolean value that indicates whether an object in the selected
configuration is the same as the object in the most current configuration (true) or
not (false).
Newer Objects in Current Configuration view
The Newer Objects in Current Configuration table lists the objects in the current version that have been added
since the selected version.
Table 89. Newer Objects in Current Configuration view
Newer Objects
Description
Object Type
Shows the type of each object, such as Hit Analyzer or Custom Field.
Object Name
Shows the display name for each of the objects.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
166
Understanding configuration modules and
objects
Configurations are divided into modules and each module contains configuration objects, as described in the
following table. An export saves all modules to a single configuration file.
Table 90. Configuration modules
Configuration Module
Objects Contained in the Module
Archiver
Archivers registered in the APM > Configure > Traffic Capture > Archivers
dashboard.
Capture Group
Capture groups defined in the APM > Configure > Traffic Capture > Capture
Groups dashboard.
Traffic Analysis
All options defined from the APM > Configure > Traffic Analysis dashboards.
Static Page Elements
All options defined in the APM > Configure Advanced Options > Static Page
Elements dashboard.
Server Settings
All options defined in the APM > Configure Advanced Options > Setting dashboard.
Sniffer
Sniffers registered in the APM > Configure > Traffic Capture > Sniffers dashboard.
In addition, this module includes options defined in the Private Keys, Monitored
Ports, Monitored IP Addresses, SiteMinder Servers, Subnet Filters, and HSM
Device dashboards.
When you import a configuration, you can select the modules you want to import. For the Traffic Analysis and
Server Settings modules, you can select individual objects rather than importing the entire module. The Import
Configuration Wizard guides you through the process.
Exporting configuration modules and objects
If you want to save a configuration permanently, you can export it. For example, you may want to save the
initial version of the configuration, a version that you use to troubleshoot web site performance issues, or a
current backup for disaster recovery. For more information, see “Exporting configurations” in the online help.
You can export your configuration to an XML file or an archive file, depending on your needs.
Table 91. Exporting configuration modules
Option
Description
Export Configuration XML
Exports the full system configuration to an XML file. You can use this type of
export to later import selected objects from one system into another. For
example, a complex sequence analyzer and its associated hit analyzers can
exported from one system and then imported into another.
Export Configuration
Archive
Exports the full system configuration plus uploaded resource files for the
following objects: private keys, HSM configuration details, static page
elements, and user agent rule files. The configuration and resources are saved
in an encrypted, binary format (readable only by the import process) and
protected by a user-defined passphrase. You should use this type of export
when you want to replicate the current system configuration, such as when you
are preparing to replace an appliance or if you want a backup of the
configuration for disaster-recovery purposes.
Restoring and importing configurations
You can roll back to a previous configuration or import a previously saved configuration. You can also import hit
analysis settings from Foglight Experience Viewer. The Import Configuration Wizard leads you through choosing
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
167
the modules you want to restore. For more information, see the “Restoring” and “Importing” topics in the
online help.
Reviewing warnings
To review warning messages, click Warnings. Warnings advise you of inconsistencies within your configuration,
but they do not prevent you from monitoring your web traffic. For example, a custom field that is not updated
by any analyzers results in a warning at the time it occurs.
Figure 107. Validation Messages dialog box
If you choose to continue without resolving the issue, the warning is saved in the list of configuration warnings.
Figure 108. Configuration Messages dialog box
You can use the list of configuration warnings to remind yourself of tasks you need to do. For example, you may
need to add an analyzer to update this custom field, or you may need to remove a custom field that is no longer
needed (such as when an analyzer is deleted). Warnings remain until they are resolved using the Traffic Capture
or Traffic Analysis dashboards.
Creating support bundles
When you contact Dell Support about appliance health issues, you may be asked to generate a support bundle
and send it to them. A support bundle contains appliance health metrics and other information about all the
appliances in your installation.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
168
Figure 109. Support Bundle dashboard
TIP: You can generate a support bundle for a single appliance using the appliance’s Console program. For
more information, see Foglight APM Installation and Setup Guide.
To create support bundles through Foglight:
1
On the navigation panel, under Dashboards, click APM > Support > Support Bundle.
2
Click Create Bundle.
It may time some time to generate and download the support bundle ZIP file.
3
Review the ZIP file to ensure no sensitive data is present.
4
Send the ZIP file to the Support Engineer assigned to your case.
Updating appliance software
You can download Foglight APM software updates from the Support Portal. When you upload the update package
into Foglight, all Foglight APM Appliances are updated at the same time.
Figure 110. Update Appliance Software dashboard
NOTE: The Management Server may be restarted at the end of the update process. If so, you need to log
in to the Foglight browser interface again.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
169
To install an update package:
1
Obtain a software upgrade package (.pkg) file from the Support Portal and save it to your local disk.
2
On the navigation panel, under Dashboards, click APM > Support > Update Appliance Software.
3
Click Update Appliance.
The Update Appliance Software dashboard opens in a separate web browser window.
4
Click Begin New Update.
5
In the Choose Appliance Upgrade Package File to Upload (.pkg) dialog box, browse to and select the
software update package that you saved to your local disk, and click Start Upload.
The update file is uploaded to the Archivers and Sniffers in your installation. A message box confirms
when the operation is complete.
6
Click Start Upgrade.
NOTE: The Management Server may be restarted at the end of this process.
The update process begins. Foglight keeps a log of the process. Monitor the upgrade status by reviewing
the on-screen log.
If the upgrade is successful, the last line of the log should state:
SUCCESS | Upgrade succeeded.
If the update is unsuccessful, review the full log to understand why the update may have failed.
7
Click Finish to return to the Update Appliance Software dashboard.
8
To review the full log, click the View Details link on right side of the dashboard. If you need help
resolving any upgrade issues, contact Dell Support.
Restarting the embedded Agent
Manager
Dell Support Engineers may instruct you to restart the embedded Agent Manager, which resides on the same
appliance that hosts a Management Server. The Agent Manager manages connections and services for the agents
collecting data in a monitored environment.
NOTE: This procedure is for embedded Agent Managers only. For remote Agent Managers, see the Foglight
Agent Manager Guide.
To restart the embedded Agent Manager:
1
On the navigation panel, under Dashboards, click APM > Support > Embedded FglAM.
2
Click Restart Service.
The Agent Manager process is shut down and restarted. This may take some time.
Foglight APM 5.9.9 Administration and Configuration Guide
Maintaining and troubleshooting
170
A
Appendix: Building regular
expressions in Foglight
Regular expressions are used throughout Foglight. For Traffic Analysis, wherever you see an Extraction
Expression option in a dialog box, you can use a regular expression.
For more details, see the following topics:
•
What is a regular expression?
•
Where can I find regular expressions?
•
Regular expression basics
What is a regular expression?
A regular expression is a sequence of characters used to describe text ranges, patterns, and various kinds of
special conditions. Regular expressions are supported in many programming languages, including Java™ and
Groovy, the core languages that Foglight uses.
The syntax of Groovy regular expressions comes from Java, so the syntax of Java and Groovy regular expressions
is the same.
Where can I find regular expressions?
Regular expressions can be found in many parts of Foglight. For example, credential mappings use the regular
expression syntax to select the hosts accessible with a specific credential.
Figure 111. Edit Resource Mapping Condition dialog box
The Excluded Drives property of the WindowsAgent is another example of a regular expression usage in Foglight.
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Building regular expressions in Foglight
171
Figure 112. Agent Properties dialog box
In addition, regular expressions can be found in LogFilter expressions, FxV Hit Analysis, and other parts of
Foglight.
Regular expression basics
The simplest regular expression is a basic text string containing only alphanumeric characters. For example,
Host1, matches the same value, Host1.
Figure 113. Example of a simple regular expression
A regular expression is fundamentally different from a pattern using file name wild cards such as an asterisk ‘*’
or a question mark ‘?’. For example, issuing a search in Windows Explorer using *.jpg as a filter matches all
files with the .jpg extension.
Figure 114. Searches using Windows Explorer
This is not a regular expression, it is a simple filter that uses wild cards. An equivalent regular expression that
selects all files with the .jpg extension is .*\\.jpg. The details of this regular expression are covered next:
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Building regular expressions in Foglight
172
•
Building a simple pattern
•
Building a pattern that matches a specific character
•
Building a pattern that matches multiple characters
•
Using advanced quantifiers
•
Using special characters and regular expression flags
•
Grouping elements in a pattern
•
Additional information
Building a simple pattern
In this section we look at a simple regular expression pattern that selects all hosts whose names start with
Host. The completed expression is Host.*. This expression contains two special characters:
•
A period ‘.’. This is a special matching symbol that matches any single character.
•
An asterisk ‘*’. This is a quantifier that instructs the regular expression interpreter to match the previous
pattern zero or more times.
The final expression, Host.*, results in matching any strings that start with Host and match any of the
following host names:
•
Host01
•
Host-test
•
Host
However, the following host names do not match this expression:
•
MyTestHost1
•
ProductionHst-9000
We now take a look at a simple pattern that matches a group of similar text strings. A common pattern for
selecting Windows® drive names is C:.*. A typical usage of this expression is in the Excluded Drives property of
the WindowsAgent.
Figure 115. Building a simple pattern
Unlike in a simple file matching filter, C:*, the equivalent regular expression requires a period between the
colon ‘:’ and the asterisk ‘*’, as in C:.*.
CAUTION: Failing to include a period in this particular example can result in data loss.
Next, we build a regular expression that selects all hosts whose name include the string Host, not just the ones
that start with Host. To do that, simply add a prefix to the above expression, Host.*, resulting in .*Host.*.
This expression matches any hosts that include Host, but not necessarily begin with it. That is because the
prefix .* translates to any combination of zero or more characters appearing before the string Host. The
expression matches each of the following host names:
•
HostProd01
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Building regular expressions in Foglight
173
•
Host-test
•
Host
•
MyHost
•
DBHost-9000, but not DBHst-9000
Building a pattern that matches a specific
character
A regular expression pattern can be used to match a string that includes a specific character. For example, you
can build a pattern that matches only the drive letters C through H. To do this, you have two options:
•
Specify the entire set of valid choices enclosed in square brackets: [CDEFGH]
•
Specify a range using the first and last character only: [C-H]
Choosing the second option, the resulting regular expression is: [C-H]:.*. This means, any letter in the range
and including ‘C’ through ‘H’, followed by a colon ‘:’, and optionally by more characters.
Building a pattern that matches multiple
characters
Using regular expressions you can define a pattern to match a multi-character pattern. For example, you can
write a regular expression to match all hosts whose names contain Host and are followed by exactly two digits.
That means you want to match the following strings:
•
Host01
•
Host03
But not:
•
Host01-bckp
The expression that matches this pattern is: Host[0-9][0-9].
TIP: Because there is no trailing .* in the expression, the host name with the -bckp suffix does not match.
Going further, you can write an expression that matches all hosts whose names contain Host, followed by
exactly two digits, and optionally a lowercase letter. That means you want to match the following strings:
•
Host01
•
Host03a
But still not:
•
Host01-bckp
The expression that matches this pattern is: Host[0-9][0-9][a-z]?.
NOTE: The question mark ‘?’ in regular expressions is also a quantifier, and not a single character wild
character like in some file systems.
Using advanced quantifiers
In addition to asterisk ‘*’ and question mark ‘?’, there are additional quantifiers that are supported in regular
expressions:
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Building regular expressions in Foglight
174
•
The plus sign ‘+’ means one or more times. For example, case[0-9]+ matches case1, case12345,
but not simply case.
•
The braces “{}” express a range. For example:
•
{3} occur exactly three times.
•
{2,4} occur at least two times and as many as four times.
•
{3,} occur at least three times up to infinity.
You can use this notation instead of the common quantifiers:
•
An asterisk ‘*’ has the same meaning as {0,}.
•
A plus sign ‘+’ has the same meaning as {1,}.
•
A question mark ‘?’ has the same meaning as {0,1}.
Using special characters and regular expression
flags
Special characters in regular expressions the regular expressions further extend their flexibility in advanced use
cases.
•
The caret ‘^’ reverses the meaning of a regular expression element. For example, [^KLM] matches a
single character that is not K, L, or M.
In the Windows® drive mapping example, the following expression excludes all drives except K, L, or M:
(?x)[^KLM]:.* # Exclude all drives except K thru M
•
The backslash ‘\’ is an escape character. It indicates that the next character is a special character and
interpreted literally.
•
A backslash ‘\’ followed by a lowercase ‘d’, \d, means a digit.
•
A backslash ‘\’ followed by a lowercase ‘w’, \w, means a word character (an alphanumeric character or
an underscore ‘_’). It has the same meaning as [a-zA-Z_0-9].
•
A backslash ‘\’ followed by a lowercase ‘s’, \s, means a white space character. It has the same meaning
as [\t\n\x0b\r\f].
•
Two backslashes “\\” followed by a period ‘.’, \\., means a literal dot.
•
The flag (?i) makes the regular expression case insensitive. In the Foglight APM, the Add OS Monitor
wizard uses this flag in the Resource Mapping regular expression. For example: (?i).*host.*.
•
The flag (?x) allows you to add comments to explain a complex or unusual pattern. The comment starts
with a number sign ‘#’. For example: (?x)[KLM]:.* # We dislike drives K through M.
Grouping elements in a pattern
The regular expression syntax provides a way to group elements together and use groups together with other
operators, as required.
•
Use parentheses “()” to group elements together.
•
•
Use a common quantifier with a group.
•
•
For example, to match a period ‘.’ followed by one or more alphabetic characters, write
(\\.[A-Za-z]+).
For example, to match the above group ((\\.[A-Za-z]+)) zero or more times, write (\\.[AZa-z]+)*.
Use a pipe ‘|’ with a set of alternatives.
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Building regular expressions in Foglight
175
•
For example, to use a prefix that is either dev or prod, write (dev|prod).
Additional information
For more information and more advanced examples, see the following topics:
•
“Appendix: Regular expressions” in the Installing the Java EE Technologies Management Capabilities
guide
•
“Appendix: Java regular expressions in FxV hit analysis” in the Foglight Experience Viewer User Guide
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Building regular expressions in Foglight
176
B
Appendix: Traffic analysis processing
When developing analyzers that rely on the actions of other analyzers or scripts, you need to ensure values are
set before another task requires that value. For example, let’s say you have a hit analyzer that sets the value of
a session custom field and another hit analyzer that uses the value of that field in a script to perform a
calculation. A script cannot know if the field has been updated for the current hit, but you can—if you
understand what happens at each stage of traffic analysis.
For more information, see the following topics:
•
Understanding the traffic analysis workflow
•
Stage 1: Run hit analysis
•
Stage 2: Run sequence analysis
•
Stage 3: Finalize stopped sequences
•
Stage 4: Finalize the session
•
Example of interdependent analyzers
Understanding the traffic analysis
workflow
When an Archiver receives a hit from a Sniffer, the Archiver extracts the hit details from the hit (which is sent
to the Archiver in a proprietary hit format) and saves those details in memory within a Java™ object. It then uses
the hit details as input to the traffic analysis process.
Traffic analysis processing occurs over multiple stages. Hit analysis occurs for every hit and sequence analysis
occurs for every sessionized hit. Recall that a sessionized hit is a hit that belongs to a particular user session
(indicated by a unique session identifier). During these analysis stages, if an action stops any active sequences
or the current session, the sequence or session is flagged as stopped. Then, after the sequence analysis ends,
final actions are run on the flagged sequences and session.
At the end of the traffic analysis process, the Archiver reviews the sensitive data storage policies option in all
matching hit analyzers. If any policies specify that sensitive data should not be stored, the Archiver applies the
policy to the collected hit data before storing the hit details and data in the Archiver database.
The following flowchart provides an overview of the traffic analysis process. The subprocesses in this flowchart
are expanded upon in the rest of this appendix.
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Traffic analysis processing
177
Figure 116. Traffic analysis process overview
Stage 1: Run hit analysis
In the hit analysis stage, all hit analyzers are evaluated to see if their match conditions match the hit. First, the
Archiver evaluates analyzers with the phase set to Filtering, because a match here causes the hit to be
discarded (no hit details are stored) and ends the traffic analysis process.
TIP: If you do not want sensitive data in a hit exposed to any kind of processing or storage, you can use the
Filtering phase to drop the hit before it is available to any other analyzers.
If the hit passes the Filtering phase, the Archiver then evaluates all Main phase hit analyzers and compiles a list
of matches. For each matching analyzer, it then evaluates warning and error conditions and sets the final
analyzer status to the worst status. For example, if a warning condition sets the analyzer status to Warning, but
an error condition in the same analyzer sets the analyzer to Error, then the final analyzer status is Error,
because Error is worse than Warning. Then, for each matching analyzer, the Archiver runs the actions
associated with the final analyzer status—with one exception. If a hit analyzer sets hit storage restrictions (that
is, sensitive data storage policies), these restrictions are deferred until after the end of the traffic analysis
process (see the flow chart in Understanding the traffic analysis workflow).
The Archiver repeats this process for Final phase hit analyzers. With these two phases, you can be sure that if
you set the value of a session-scoped custom field in the Main phase based on the final analyzer status, its value
can be reliably used by a Final phase hit analyzer.
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Traffic analysis processing
178
Figure 117. Hit analysis stage
After running the actions for matching analyzers in all phases, the Archiver determines the final hit status for
the hit. The final hit status is the worst final analyzer status from among all the matching hit analyzers. The
Archiver then iterates through the list of matching hit analyzers and runs any actions associated with the final
hit status.
NOTE: Do not confuse final analyzer status with final hit status. The final hit status is determined at the
end of the hit analysis phase, after analyzers in all phases are evaluated. If a Main phase hit analyzer
updates a custom field based on the final hit status, the value will not be set (and therefore is not
available) until after the Final phase hit analyzers are evaluated.
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Traffic analysis processing
179
Stage 2: Run sequence analysis
If the hit belongs to a session, the Archiver evaluates sequence analyzer events and special events. A hit may
belong to one session and zero or more sequences.
The analysis starts by evaluating events that can start sequences and end the session, that is, sequence events
with an entry state of Not Started and the Stop Session Event special event. For each match, the Archiver runs
the actions defined (if any) for the event. Typically, for a sequence event, this means starting a sequence, but it
can also set the sequence status, set custom fields, and run scripts. If this hit matches an event that ends the
session, the Stop Session flag is set as well as Stop Sequence flags for all active sequences within the session.
NOTE: While an event can update a metric in memory, the metric is not stored in the database until the
finalize sequence stage. For more information, see Stage 3: Finalize stopped sequences.
Next, the analysis evaluates active sequences and events that can end sequences. An active sequence is any
sequence that has been started and has not yet been finalized, including sessions and sequence flagged as
stopped. If there are no active sequences, the Archiver ends the sequence analysis. Otherwise, the Archiver
evaluates sequence events with an entry state of Active and the Stop Sequence Event special event against all
active sequences, including any sequences started in the previous phase. For each matching event, the Archiver
runs the actions defined (if any) for the event. If an event ends a sequence, a Stop Sequence flag is set.
Finally, if there are sequences flagged as stopped in the previous two stages, the Archiver evaluates all
sequence analyzer events where the entry state is Stopped. If any event conditions match the hit, the Archiver
performs the actions associated with the event.
NOTE: It is possible for a hit to start and stop a sequence during the sequence analysis stage. In this case,
you get a sequence with one hit.
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Traffic analysis processing
180
Figure 118. Run sequence analysis
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Traffic analysis processing
181
Stage 3: Finalize stopped sequences
When the Stop Sequence flag is set for one or more sequences, the Archiver iterates through the list. For each
stopped sequence, it sets the sequence analyzer status, sets storage policies and updates metrics based on the
sequence analyzer status, and writes the sequence data to the database.
NOTE: All other actions—such as, updating a custom field, executing a script, setting the status, starting
or stopping the sequence, and stopping the session—are performed when the event conditions are
evaluated during the sequence analysis stage.
Figure 119. Finalize stopped sequences
Stage 4: Finalize the session
When the Stop Session flag is set, the Archiver evaluates session analyzers. It sets a final analyzer status for
each analyzer and runs the actions associated with that status. When the session analyzers are resolved, the
Archiver determines the final session status and runs the actions associated with that status. After the session is
resolved, the Archiver writes session details and any updated session data to the database.
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Traffic analysis processing
182
Figure 120. Finalize the session
Sessions can also end due to a session timeout. The end-of-session steps are the same as those shown here.
Example of interdependent analyzers
Let's say you have four different login pages depending on the type of user or location of user (internal versus
external, new versus existing). You want to track information about that user, such as the user’s name, that is
specific to the login page they used. While the name may not be specific to the login page, how you extract the
name from the hit details may be tied to the login page.
You can set up four hit analyzers to count the number of logins of each type and save the name of the user in a
session custom field. However, you also want to count the total number of times a login page was accessed. You
can set up a Final phase analyzer to count the number of times any of the four hit analyzers matched.
Foglight APM 5.9.9 Administration and Configuration Guide
Appendix: Traffic analysis processing
183
About Dell
Dell listens to customers and delivers worldwide innovative technology, business solutions, and services they
trust and value. For more information, visit http://www.software.dell.com.
Contacting Dell
For sales or other inquiries, visit http://software.dell.com/company/contact-us.aspx or call 1-949-754-8000.
Technical support resources
Technical support is available to customers who have purchased Dell software with a valid maintenance
contract and to customers who have trial versions. To access the Support Portal, go to
https://support.software.dell.com.
The Support Portal provides self-help tools you can use to solve problems quickly and independently, 24 hours a
day, 365 days a year. In addition, the Support Portal provides direct access to product support engineers through
an online Service Request system.
The Support Portal enables you to:
•
Create, update, and manage Service Requests (cases).
•
View Knowledge Base articles.
•
Obtain product notifications.
•
Download software. For trial software, go to http://software.dell.com/trials.
•
View how-to videos.
•
Engage in community discussions.
•
Chat with a support engineer.
Foglight APM 5.9.9 Administration and Configuration Guide
About Dell
184