Simplifying Event Log Management
www.veriato.com
Simplifying Event Log Management
So you have servers. Probably lots of them – could be three, ten, a hundred or more. Each of them contains a wealth
of information about the security, performance and reliability of your users, servers and the network they reside on.
Event Logs are the window into what’s happening on your servers. They are commonly used for:
• Identifying behaviors
• Ensuring security
• Finding problems
• Proving compliance
• Quantifying performance
But when you think of Event Logs, you don’t think about how awesome they are to point out and help solve all the
issues implied above. And we all know why – Event Logs are a pain.
Why are Event Logs So Painful?
Let’s start with the obvious answer: There’s an inordinate amount of data. According to Gartner, a medium
- sized enterprise creates 20,000 messages per second of operational data in activity logs. In a single, 8-hour
day this comes to 500 million messages, adding up to more than 150 GB of operational data.
Now, this may not accurately represent your environment, but it evokes the same emotions you already feel
when you think about your servers and all the logs they contain (and your head begins to spin).
The second obvious answer: Finding the needle in the proverbial event log haystack. Make that haystacks
– you’re responsible for monitoring multiple servers, multiple logs, and multiple events. And once you have
a grasp on all the data you need to search through, you need to determine what are you supposed to look for
- Is it the event ID, the description, the source? Which query will provide a meaningful result?
To put all of this in perspective, let’s look at five aspects of Event Log Management that need to be addressed.
Consolidation
Unless you like doing the same job repeatedly for each server you manage, you’re going to need to consolidate
your logs into one location. This makes the remainder of the Event Log Management tasks far easier.
Questions addressed here usually include:
• Which server logs should I (and which should I not) consolidate?
• Do I want/need to consolidate every log?
• Do I want/need to consolidate every event entry?
2 | www.veriato.com
A medium - sized enterprise
creates 20,000 messages per
second of operational data
in activity logs.
Management
This next aspect sounds a bit redundant (of course there’s management in Event Log Management), but what is
meant here is the management of the data that is consolidated. Storage, retention, backups, further consolidation
all need to be addressed from several standpoints, including security and compliance. Questions addressed here
usually include:
• Where will I store the actively used logs?
• How long will I maintain my log data?
• What is my archiving strategy (think both age and medium)?
Monitoring
Deciding what to monitor is always a challenge. What gets monitored usually depends on what is important to a
business. If it’s security or compliance, the answer may be access to data, or account creation in Active Directory, or
even logon failures. If server performance, it may be Exchange service errors, or operating system warnings. Again,
it all depends.
Microsoft provides a number of fields to search on, so we’re not just talking about searching for one field. Table 1
shows a sample of the fields you can use to filter your data.
Event ID
Date/Time Logged
Source
Task Category
Log Name
Keywords
Level
Computer Name
User
Description
Table 1: Windows 2012 Event Log Fields
But wait, there’s more! Monitoring isn’t about a single event. The simple copying of a file will
generate a myriad of entries. So it’s not always as easy as “show me the event where” but may
involve defining how a number of entries correlate to represent the event you wish to monitor.
Alerting
Monitoring, by itself, is useless. Without telling someone the house is on fire,
the house will simply burn to the ground. The same is true with Event Logs.
It’s great to monitor for specific issues, events, actions, etc, but it is the alerting
that puts IT into action. Traditionally, alerts take the form of an email, but can
also be SMS texts, SMNP traps, Dialog Boxes, even sounds.
Monitoring, by itself, is useless.
Without telling someone the
house is on fire, the house will
simply burn to the ground.
Remediation
IT folks are some of the most dedicated, hard-working folks, such that they don’t always have time to fix every issue
exactly when it happens. Part of your Event Log Management strategy should be the automatic fixing (or at least the
first attempt to fix) an issue. This can be reboots, restarts of services, running of scripts that perform actions and the
like.
3 | www.veriato.com
Event Log Management Simplified
Your Event Log Management strategy should include all 5
aspects: Consolidation, Management, Monitoring, Alerting
and Remediation.
Microsoft Provides the Basics
Microsoft does give credence to the idea that log management isn’t easy, which is why their Event Viewer (shown in
Figure 1) has undergone changes throughout the years to include not just the ability to Find and Filter events, but
also to perform some basic remediation.
Figure 1: Windows Server 2008’s Event Viewer
Windows Server allows you to select a specific event and perform one of three actions
(shown in Figure 2), should the event occur.
4 | www.veriato.com
Figure 2: Attaching a Task to an Event
Event Forwarding
With Windows Server 2008, Microsoft introduced Event Forwarding, shown in Figure 3.
Figure 3: Event Forwarding Subscription Properties
With Event Forwarding, logged events on Windows Server 2003 and 2008 servers can
be forwarded to a centralized server, based on specified criteria, as shown in Figure 4.
5 | www.veriato.com
Figure 4: Specifying events to be forwarded
The setup of Event Forwarding is a bit of work. It involves a number of steps just to get the servers ready to forward
events and then you need to configure what gets forwarded. Truly, it is a good attempt at helping with the challenge
of managing multiple servers by consolidating events, but is intended to scale to a few servers at most and only
addresses one of the five parts of Event Log Management – Consolidation.
Not Making the Grade
Table 2 shows a representative measurement of how Microsoft’s native tools address the five aspects of Event Log
Management.
Basic Event Logs
w/Event Forwarding
Consolidation
None
Yes, not scalable
Management
None
None
Monitoring
Per event
Per event
Alerting
Basic
Basic
Remediation
Basic
Basic
Table 2: Measuring Native Tools
Don’t forget your other sources!
To make log management even more complex, Microsoft Windowsbased servers
are only one source of logs you need to manage. Your non-Windows servers,
firewalls, printers, switches, etc. all have valuable information to provide about
the security or performance (or both) of your networks. There are other sources
you need to include:
6 | www.veriato.com
Monitoring, by itself, is useless.
Without telling someone the
house is on fire, the house will
simply burn to the ground.
•
•
Syslogs – This is the most common standard for logging outside of Microsoft. Syslogs utilize a push
technology that require a service running somewhere to accept and consolidate the syslog data.
Text Logs – Additionally, some systems, including SQL Server, write to text-based log files.
These should also be considered.
Event Log Management Simplified
Your Event Log Management should include monitoring all
relevant sources on (and in some cases, off ) your network and
not just Windows
It’s Still Not Easy Enough. Now What?
Reducing the amount of work needed to manage Event Logs can only be accomplished by utilizing a third-party
solution designed to do the work you’d be doing manually, or with limited automation with native tools.
Meet Veriato Server Manager
Let’s discuss how to make Event Log Management easier by discussing three key aspects you should find in a thirdparty solution and by introducing you to Veriato Server Manager, shown in Figure 5.
Figure 5: Veriato Server Manager
7 | www.veriato.com
Server Manager consolidates, monitors, alerts on and responds to critical events, providing centralized management
and reporting of Event Log and Syslog data. Server security and performance is maintained, the health of server
resources is monitored, and adhering to compliance standards can be proved.
To truly consider Event Log Management “simple’, the solution you use should meet the followingthree criteria:
• Scalable
• Centralized
• Automated
Let’s look at each and how Server Manager meets each.
Scalable – Single Solution
We’ve already discussed scalability a bit in this whitepaper in the context of Event Forwarding.
But your work encompasses multiple logs, multiple servers, and multiple types of logs.
Server Manager provides robust capabilities to address Microsoft Event Logs, Syslogs and Text Logs from within the
same solution, allowing you to consolidate, manage, monitor, alert and remediate issues across your entire network.
Figure 6 shows Server Manager’s comprehensive support for consolidation, monitoring and management of all three
log types.
Figure 6: Server Manager’s Log Templates
Some of you are monitoring logs for security reasons, while others are monitoring to maintain
performance levels of service. If uptime and performance are of concern, you need to be
monitoring beyond just logs. Server Manager also monitors server resources, disks, applications,
Windows services, databases, TCP ports, well-known web services, you name it – all under one
roof so you get a comprehensive view into what’s going on from both the log and performance
perspective. Figure 7 shows the various types of performance-related monitors Server Manager
supports.
8 | www.veriato.com
Figure 7: Server Manager monitors all aspects of performance
Event Log Management Simplified
If performance is a concern, having a single solution that
monitors both a servers logs, as well as its resources, services,
processes, etc. provides you with a comprehensive view into
server performance.
Scalable – Multitudes of Nodes
Server Manager was designed to support the monitoring needs of your network. It can simultaneously be monitoring
your Windows servers, Unix boxes, workstations, SANs, NASs, routers, printers, hubs, switches, firewalls, appliances,
websites and more.
Scalable – Template Driven
Given that Server Manager can monitor so much, it has been designed to simplify the aspects of
monitoring so that you’re not repeating the same tasks over and over again.
Server Manager utilizes templates, shown in Figure 8, to define the various aspects of monitoring
and management of event logs, including:
• Computers to monitor
• Events to monitor
• Frequency
• Actions
9 | www.veriato.com
Figure 8: Examples of Server Manager’s template technology
Let’s use at a real-world example to see how this benefits you. If you monitor multiple Exchange or SQL servers, you
can simply define the events that need to be monitored, the times of day to monitor and the actions to take when the
monitors are triggered then quickly apply that same template to all of the servers, as is appropriate. Likewise, should
you simply want to reuse one aspect of that definition – let’s say an action to be taken – and apply it to a completely
different set of servers being monitored for a completely different set of events, you can take that action template
and utilize it somewhere else.
Next, let’s take a look at how a centralized solution simplifies Event Log Management.
Centralized – Log Consolidation
To properly monitor and manage logs, they need to be in one place.
With Server Manager’s Consolidation Template, shown in Figure 9,
you can easily select the servers, logs to be consolidated and filter the
consolidated events (Figure 10) to ensure you only collect the events
you need. Post-consolidation actions can also be applied to the logs as
they are pulled in, providing you with management and alerting the
moment consolidation occurs.
10 | www.veriato.com
To properly monitor and
manage logs, they need
to be in one place.
Figure 9: Template-based Event Consolidation
Figure 10: Event Filtering
11 | www.veriato.com
Event Log Management Simplified – Templates
Remember that because Server Manager is template-driven, an
Event filter, like the one shown in Figure 10, can be reused for
additional consolidation, views of logs, and reporting.
Centralized – Log Management
Once you have the data, you need to plan on how you will store it, back it up, and make it available (as is appropriate)
for review, reporting and retention.
Server Manager supports storing consolidated Log data in 4 different mediums:
• SQL Server
• Oracle
• MySQL
• Server Manager’s own proprietary binary file format
Besides backing up any consolidation databases, you may need to archive Logs directly for security or compliance
purposes, including being encrypted and digitally signed. Figure 11 provides an example of how Server Manager can
be configured to automatically backup Logs, which can be scheduled using Server Manager’s Schedule templates.
Figure 11: Event Log Backups
12 | www.veriato.com
Centralized – Log Reporting
The beautiful part about event log management is you already use filters. And what’s the basis for log reporting?
Filters of course! So building reports is as easy as creating a filter template (or reusing one that you’ve already created).
Server Manager has 15 turnkey reports (a sample Failed Logons report is shown in Figure 12), but is designed to allow
you the flexibility to quickly generate your own reports using the report templates.
Reports can easily be re-run against current data, scheduled, posted to websites for viewing and
saved out to HTML, TXT or CSV formats.
Figure 12: Reporting with Server Manager is as easy as creating a filter
Automated – Response
If Log Management stopped here, you’d be completely up to your ears in properly consolidated and monitored logs
and with an Inbox full of alerts awaiting your response. Server Manager provides you with the ability to respond
using a wide variety of actions, shown in Figure 13.
The actions fall into three categories:
•
Alerts – Utilized to make the appropriate staff aware of an issue. These include sending an email or text,
displaying a message box and playing a sound.
•
Documentation – Used to record the occurrence of an event in a separate system, log, etc. These include
writing the event to a database, another event log, a syslog server, a file and sending an SNMP trap.
•
Remediation – Used as “first response” to fix issues. These include managing Windows services (stop, start, restart) and launching a process (which opens up a wide variety of possible actions – running a script,
launching a backup or restore, shutting down a server, etc. – the list is limitless).
13 | www.veriato.com
Figure 13: Server Manager provides a number of actions to respond to events
Automated – Log Management
To make Event Log Management truly work, it needs to be “set it and forget it.” Server Manager, as a whole, meets this
criterion. It was designed as a management platform that performs the actions you’d normally accomplish by manual
means, and automates each aspect of Event Log Management. From consolidation, to management, to monitoring,
to reporting, to responding, Server Manager does it all automatically.
Figure 14: Server Manager completely automates Event Log Management
14 | www.veriato.com
Does Server Manager Make the Grade?
Let’s bring back the grading we gave to Microsoft’s native tools and see how Server Manager stacks up in Table 3.
Server Manager
Basic Event Logs
w/Event Forwarding
Consolidation
Yes
None
Yes, not scalable
Management
Yes
None
None
Monitoring
Yes
Per event
Per event
Alerting
Yes
Basic
Basic
Remediation
Yes
Basic
Basic
Table 3: Seeing how Server Manager measures up
What Else Does Server Manger Offer?
There’s a reason the product wasn’t given a name that implied management of event logs only. Remember, in the
area of Log Management, it also manages Syslogs and Text logs, giving Server Manager the ability to monitor just
about any system that produces logs.
Additionally, as was briefly mentioned in the Scalable – Single Solution section of this whitepaper, Server Manager
also monitors, alerts and remediates issues for:
•
Windows Resources – includes memory use, CPU utilization, and network throughput, individual
processes, services, Active Directory and clock synchronization
•
Network Resources – includes email and web services, SSL and domain expiration
•
Disk Resources – includes disk space, SMART status, directory sizes, and file counts
•
Database Resources – includes SQL Server, Oracle, MySQL and ODBC
15 | www.veriato.com
Conclusion
Your efforts to manage Event Logs can be with put towards an arduous and tedious process of systematically going
through logs, looking for issues, and taking the appropriate action each and every time, or you can make a one-time
investment to set up an automated way to stay informed of and maintain the current state of your Windows
environment. With Veriato Server Manager, Event Log Management becomes a simple task of establishing
once what needs to be monitored (and what to do about it), reducing issue elevation, increasing server and
service uptime and improving your productivity.
Server Manager Resources
Free Trial
Go to www.veriato.com for a Free Trial or email us at: [email protected]
Corporate Offices
Veriato, Inc.
4440 PGA Boulevard, Suite 500,
Palm Beach Gardens, FL 33410
1.888.598.2788
1.772.770.5670
International
United Kingdom
C2, Dukes Street
Woking
Surrey, GU21 5BH
+44 1483 397744
www.veriato.com